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DIAGNOSTIC AND REMEDIAL STEPS IN READING! 


onal Ins WILLIAM S. GRAY 


baton, Ma ‘ 

The University of Chicago 

“ye Any report which is made at this time of diagnostic and reme 
= dial work in reading must be tentative and incomplete for two rea 

hers’ sons. The technic of diagnosing individual and group needs has 

al nl not been fully developed. Furthermore, appropriate remedial de 


mm vices have been organized for only a limited number of disabilities 
mart, @ It is possible, on the other hand, to secure helpful suggestions from 
— ze study of specific methods now employed in various laboratories 
pweYes and school systems of the country. 

Experience has proved that diagnostic and remedial work in 
reading should begin with a study of the accomplishments of all 
the pupils of aschool. This step enables teachers to determine the 
ngton. major problems of reading instruction in each grade for the year. 
It also secures information concerning the needs of every pupil. 


Unless a systematic program of testing is followed there is danger 

that many significant individual difficulties will not be noted. 
thly exc In the Elementary School of the University of Chicago at least 
four records of accomplishment in reading are secured at the 
beginning of each school year: (a) mastery of the rudiments, or me- 
chanics of oral reading, including both rate and accuracy, as meas- 
ured by the Standardized Oral Reading Paragraphs; (b) rate of 
reading simple material silently as measured by the Courtis’ Silent 
ys Reading Test, No. 2; (c) ability to understand simple passages 
as measured by Courtis’ Silent Reading Test, or Burgess’ Scale for 
Measuring Ability in Silent Reading (P.S.-1); (d) ability to under- 
stand increasingly difficult passages, as measured by Monroe’s 
Silent Reading Test, or Thorndike’s Scale Alpha 2. Although 
ornds these tests do not cover all phases of reading, they secure a 
1 A paper presented at the meeting of the National Association of Directors of 

, Arti Educational Research at Atlantic City, N. J., March 3, 1921. 
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sufficient amount of information for an exceedingly helpful prelimi 
nary diagnosis. In the report which follows the discussions of 
diagnostic and remedial steps are limif€d on account of time to on 
phase of reading. 

As soon as the tests have been given and the scores calculated, 
a careful study is made of the average scores of each grade to de 
termine which phases of reading should be emphasized most dur 
ing the year. For illustration, Figure 1 represents the averag: 
scores in oral reading in October, 1919. The vertical lines are th 
lines on which the scores for the grades are indicated. The solid 
oblique line represents the standard scores. The dotted oblique 
line represents the scores for the University Elementary School 
in October, 1919. 

An analysis of the records revealed two significant facts: (a) oral 
reading needed little emphasis in each grade as a whole above the 
111-B, because the average class scores at the beginning of the year 
were higher than the standard scores for the end of the year; and 

b) more or less class instruction in oral reading was necessary in 
the u-B, I-A and mi-B grades because the average class scores 
were lower than the standard scores. 

The second step in the diagnosis included an analysis of the 
individual scores in each grade. Table I shows the distribution of 
the pupils of each grade in oral reading. The numbers above the 
vertical columns indicate the oral reading scores. The grades are 
indicated in the left-hand column and the total number of pupils 
in each grade is indicated in the right-hand column. The entries 
in the table show the number of pupils receiving each score. Thus 
in the II-B grade, 1 pupil made a score of 5, 1 a score of 10, 3 a 
score of 20, etc. 

The irregular vertical line in the table indicates the scores 
which pupils should make in oral reading to do most effective 
work in fluent intelligent reading. In the University Elementary 
School, these scores are 40 for the 11-B grade, 45 for the I-A grade, 
50 for the m-B grade, and 55 for all of the higher grades. The 
scores were determined three years ago after careful comparative 
studies had been made of the accomplishments of pupils in various 
phases of reading. 

The pupils in the lower grades who ranked higher than these 
scores were promoted to a higher grade, it their work in other sub- 
jects justified promotion. If they were not promoted, they were 
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4 JOURNAL EDUCATIONAL RESEARCH Vol. 4, No. ! 
excused from oral-reading exercises, and either provided with ma 
terial to read silently or given additional work in those subjects i: 
which they needed help. The pupils who scored lower than th , 
desirable standards were given help in oral reading in proportior , 
z to their needs. In attempting to organize appropriate remedia | 
. instruction for these pupils, additional diagnostic steps wer 
adopted. 
rABLE I. DISTRIBUTION OF PUPILS IN ORAL READING 
2 Grade 5) 10, 15) 20) 25) 30, 35 60 65 70, 75, Total 
q 
4 
II-B 1 3} 1) 2 2 5 1 0) 
4) 2) 2 26 
III-B 3 z | 25 
III-A 2} 28 
IV-B ; 3 3 4 3 32 
V-B 21) 2 3 
V-A 123358910 S| 1 31 
VI-B 1} 2 4 9 6 6 28 
VI-A 2) 4 213] S|} | 26 
A study was first made of the kinds of errors which pupils mak: 
in reading. The most significant errors with words which were | 
discovered are non-recognitions, total mispronunciations, an: 

partial mispronunciations of (a) monosyllabic words and (b) poly- | 
syllabic words. The most important errors with groups of words jj 
are poor phrasing, omissions, insertions, substitutions, repetitions, j 
and reversing the order of words and phrases. : 

A careful study was next made by the teachers of the methods 
of instruction which are appropriate in preventing or eliminating 
each type of error. Such matters were emphasized as thefollowing: 
the importance of numerous simple interesting reading exercises x 
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to promote the establishment of effective habits of reading; the 
value of flash-card exercises in the lower grades in deve loping a 
sight vocabulary, and in increasing the span of recognition; the 
importance of directing attention to the content of what is read 
in correcting mispronunciations; the value of phonetic analysis 
in eliminating partial mispronunciations of monosyllabic words; 
the value of a study of the rules of syllabication and accent in elim 
inating the mispronunciation of polysyllabic words; the impor 
tance of exercises in recognizing thought units in sentences in order 
to improve phrasing; and the value of careful reading to eliminate 
omissions, substitutions, insertions, and repetitions. 


KINDS OF ERRORS IN ORAL READING 


I. Worps 


1. Non recognition 
2. Total mispronunciation 
3. Partial mispronunciations 
a. Monosyllabic words 
Beginnings and endings of words 
(Usually a consonant or phonogram 
Middle parts of words 
(Usually a vowel or digraph 
Enunciation 
b. Polysyllabic words 
Syllabic ation 
Accent 
Repetition of one or more parts 
Substitution 
Omission or insertion 
Inaccurate pronunciation of a part or a syllable 


II. Wirn Groups or Worps 


1. Poor phrasing 
2. Omissions, insertions, and substitutions 
a. Which change meanings 
b. Which do not change meanings 
3. Repetitions 
a. To secure a better attack on a word or phrase 
b. To correct an error in pronunciation 
c. To verify meanings 
4. Reversing the order of words and phrases 


The study of devices for overcoming difficulties was accompa- 
nied by a detailed analysis of the causes of difficulty in oral reading 
Some of the more important causes which have been discovered 
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appear in the outline. Irregular attendance causes slow progress 
on the part of some pupils. School physicians and teachers fre- 
quently find that poor health, malnutrition, or nervous disorders 
interfere with rapid progress. Studies of the nationality of the 


. child, his home environment and training, the economic conditions 
has under which he lives and the peculiarities of the section of the 

country from which he comes have suggested remedial measures, 
| such as a larger amount of training in English to overcome lan 
q guage hi all aps, or a generous provision of interesting reading 
s all which may be read at home. Furthermore, studies of the 
a previous instruction of the child have revealed the fact that his 
w difficulties are due to inappropriate or ineffective methods of in 
" struction, to constant use of difficult selections, to an inadequate 
amount of reading material, or to uninteresting selections which 


were used. 

| The organic causes of difficulties in reading are far more diffi 
cult to detect and to remedy. Many children experience genuine 
difficulty because of visual defects which can be remedied through 


the use of appropriaie lenses. Some children find it difficult to 
form the motor coordinations which are essential in reading. Care- 
fully prepared exercises to strengthen the muscles of the eyes have 
proved helpful. <A limited visual span prevents some readers 
from recognizing a group of words at a given fixation. This limi- 
tation can be detected through the use of a tachistoscope. Short 
exposure exercises have proved effective in increasing the amount 
which one can recognize at a single fixation of the eye. A low 
degree of visual acuity frequently leads to numerous errors in 
reading. It can be detected through a letter-marking test, and has 
been improved by exercises of the same type. 

Defects in the brain tissues frequently cause difficulties in 
reading. Word blindness, for illustration, is defined as extreme 
difficulty in learning to recognize printed or written language by 

persons of normal mentality and vision. It has been overcome in 
a limited number of cases by methods of instruction which make 
a vigorous appeal to the child. Vocal defects, such as malforma- 
tions of the vocal organs and enlarged tonsils, interfere seriously 
with effective reading. Breathing irregularities, which are caused 
by adenoids or by inadequate control of the diaphragm and chest 
muscles, may lead to errors. Auditory defects, such as word or 
sound deafness, frequently lead to errors in pronunciation. 
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A third group of causes are psychological in character. They 
include genera} mental incapacity, Gnadequate attention to mean 
ing, failure to associate appropriate meanings with words) limited 
meye voice span, limited span of recognition, inability to remember 
new words easily, capacity to learn words only very slowly fot- 
getting them quickly and easily, and inability to analyze and _pro- 
nounce words effectively. 

With the various causes of difficulty in mind, the teachers made 
a careful study of the records of their pupils. Those who were 
below the required standard but who gave evidence of no unusual 
or peculiar difficulty were given oral-reading instruction appro 
priate for normal second- and third-grade classes. Special effort 
was made, however, to interest the pupils and to provide them 
with a large amount of simple interesting material for use in class 
ind for supplementary outside reading. 

The regular instruction which has just been described was 
supplemented by special exercises to meet the needs of small 
groups of pupils. For illustration, one group did considerable 
silent reading to develop habits of thoughtful reading. A second? 
group gave attention to word analysis and phonics to gain inde? 
pendence in word recognition. A third group drilled on quick, 
perception exercises to increase the span of recognition. A fourth 
group selected thought units in sentences to improve phrasing. 


amen much time was devoted to these exercises, the pupils 
were always given abundant opportunity to read in order that 
)the help which was derived from the special exercise might be re- 


Ss as soon as possible in their reading habits. 
f . . . 
rhe pupils who did not respond to this treatment were sent 


to the special “‘remedial”’ teacher for a thorough diagnosis and 
for appropriate remedial exercises. In order to describe con- 
§ cretely the diagnostic and remedial steps which were employed, 
the case of a fourth-grade boy will be presented in detail, who 
§ found it necessary in September, 1920, to give up some of his 
school work because of inability to read. This situation was so 
acute that it seemed for a time that he would have to discontinue 
school altogether. In the diagnosis of his case he was first given 
the Standardized Oral Reading Test in which he scored distinctly 
below the standard for the second grade. His record showed that 
he was unable to recognize many very simple words. He made 
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numerous substitutions, such as says, for said, was for were, sma 

for same, and he for the boy. Furthermore, he recognized words 
individually rather than in groups, which resulted in an unusuall; 
slow rate of reading. 

The most characteristic difficulty which was revealed was a 
peculiar form of confusion which led to frequent repetitions. In 
this connection he frequently pronounced words which appeared 
to the right before pronouncing those to the left. This error o 
curred so frequently that detailed consideration was given to it 
later in the diagnosis. 

The Burgess’ Scale for Measuring Ability in Silent Reading 
(P. S. -1) was given next to determine how effectively the subject 
read silently. In this test he ranked distinctly below the median 
score for the second grade. 

Courtis’ Silent Reading Test was given next to secure additional! 
information concerning the subject’s rate and comprehension in 
silent reading. In accuracy of interpretation he excelled the 
fourth-grade standard, but in rate of reading he ranked very low 
The results of this test suggested quite definitely that his major 
difhculty related to the recognition of words, rather than to their 
interpretation. 

Additional evidence supporting this conclusion was secured 
through a supplementary test based on passages of the Courtis 
rate test. After the subject had read three passages silently, he 
was asked to reproduce what he had read. He reproduced onl) 
16 words, or 12.5 percent of the amount read. He then read thre« 
passages orally and reproduced 18 percent. Three passages were 
then read to him and he reproduced 70.1 percent of the content 
The superior score which was made when the subject was relieved 
of the problem of recognition made it clear that his difficulty in 
reading related primarily to recognition rather than interpretation. 
A study was therefore made of the causes of difficulty in recogni- 
tion. In this connection four steps were taken. ‘ 

1. Jones’ Vocabulary Test was first given to determine the 
ability of the subject to pronounce words accurately. The records 
showed that he failed to see many letters unless his attention was 
called to them, he confused the sounds of important letters oi 
words, he was unable to analyze short words containing the sim 
plest phonetic elements, and was unable to recognize at sight 
frequently recurring words, such as what, that, and you. 
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sma 2. The difficulty encountered in the recognition of words sug- - 
vords | gested the possibility of marked weakness in visual memory. A a 
ually | visual memory exercise was therefore given to the subject and to - 
four fourth-grade pupils who ranked distinctly above the average a 
yas a in general intelligence, school marks, and oral reading accomplish- = 
In’ @ ment. The special subject of this study made fewer errors than 7 
‘ared 7 any other pupils. This indicated that very little if any explana- - 
r o¢ tion for slow progress in reading could be attributed to gross de- - 
to it fects in visual memory. - 
3. A study was next made of the ability of the subject to rec 
ding enize individual letters, parts of words, and groups of words at 
ject single fixations of the eye. The materials which were used in- 
dian luded the following: the 26 letters of the alphabet, 18 two-letter 
w words, and 10 each of two-letter nonsense syllables, four-letter 
onal § words, four-letter nonsefse syllables, two-word phrases, and three- 
n in word sentences. These were presented uniformly through the use 
the @ of a drop tachistoscope. Each item was presented until it was 
low accurately recognized. The four fourth-grade pupils who took 
ajor the visual memory test took this series of tests also. Table II 
heir @ contains a summary of the results. 
ired TABLE II. THE NUMBER OF ERRORS IN RECOGNITION BY FIVE 
irtis SUBJECTS IN THE TACHISTOSCOPIC EXPERIMENT 
he 
nl) Recognition Series Subject A) Subject B) Subject C Subject D/Subject E 
vere Isolated letters. . 16 is 9 20 
ent Two-letter words 5 6 10 1 2 
ved l'wo-letter nonsense syllables 7 4 2 } 1 13 
Three-letter words........ 10 0 3 
lhree-letter nonsense syllables 3 9 2 6 
Four-letter words... . ee 19 1 1 2 4 
Four-letter nonsense syllables 38 7 17 2 9 
l'wo-word phrases 32. 3 4 1 2 
Three-word sentences. 25 3 16 1 5 


Subject A made more than twice as many errors as any other 
subject. He recognized isolated letters, two-letter words, and two- 
letter nonsense syllables as accurately as the other subjects. His 
difficulties increased markedly with the three-letter units. The 
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numerous errors made in recognizing three- and four-letter non 
sense syllables indicated marked inferiority in the accurate recog 
nition of the details of a group of letters. Furthermore, the errors 
made in recognizing two- and three-word phrases and sentences 


indicated that his span of recognition was much narrower than 


that of the other subjects. 

4. Before drawing final conclusions a photographic record was 
made of the eye-movements of the subject while reading a simpk 
unfamiliar passage silently. A summary of the difficulties whic] 


were discovered follows. (a) The subject recognized each Word , 


individually. This is indicated by the fact that there is at least on 
fixation, and in some cases, several fixations, per word. (b) These 


fixations were very irregular and unsystematic. This indicates 
that the subject did not have a definite method of extricating him 
self from a word difficulty. (c) The subject used an inaccurate 


uneconomical return sweep from the end of one line to the begin 
ning of the next. This was indicated by the location of the first 
fixation of each line. In the second line it was fourth from th 
left, in the third line it was eighth, in the fourth line it was twelfth 
and in the fifth line it was seventh. (d) Instead of beginning at the 
left and going forward step by step, the eye skipped about, some 
times fixating on a point very much ahead of where it should be 
and at other times moving to the left over parts which had already 
been read. A partial explanation for this irregularity is*found in 
a statement made by the subject. /He stated that in reading a sen 
tence he tried to find a sufficient number of words which.he knew 
to enable him to guess at the meaning of the rest. This resulted 
in irregular wandering movements rather than definite progressive 
movements and made fluent reading more or less impossible. 

The diagnosis which has beermreported thus far revealed five out- 
standing defects in the reading habits of the subject: (a) inappro- 
priate motor habits in making the return sweep; (b) irregular 
progression of attention from left to right; (c) failure or inability 
to scrutinize words in sufficient detail to’ recognize significant 
parts; (d) inability to analyze new words; and (e) inability to rec- 
ognize words in groups or thought units. 

The remedial program of instruction which was adopted in- 
cluded three distinct features. For thirty minutes each day the 
subject was under the immediate supervision of Miss Delia Kibbe, 
a special teacher of unusually bright and slow children in the Uni- 
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‘non- versity Elementary School. The problem assigned to Miss Kibbe 
‘ecog is the dev elopment of ability to recognize words independently 
Tors During a second period of thirty minutes the subject was under 
ences ‘he direction of the writer. An attempt was made (a) to correct 
than the inappropriate motor habits which were en iployed in making 
e return sweep; (b) to promote regular progression of attention 
1 was from left to right; (c) to increase the ability of the subject to note 
mpl -mportant details of words; and (d) to increase the span of recog- 
vhich © nition. 
word , In order to develop effective eye-movements interesting selec- 
t one | | tions were typewritten in three forms. In ea h type the lines were 
Phese trated more widely than in ordinary print in order to promote 
cates ra apid and accurate return sweep. The subject was instructed 
him to move his eyes quickly from the end of one line to the be ginning 
rate of the next without stopping to look at any of the words of the 
egin- ™ second line. After ten five-minute exercises in which this problem 
first was emphasized, he was able to make the return sweep effec tively. 
1 the [he words of the three types of exercises were typewritten in 
lfth. ©) modified form in order to promote regular progression of attention ee 
t the from left to right. In the first type the words were written five- a 
ome etter spaces apart. The subject was instructed to read the words - 
id be in order without glancing to the right or to the left of a given word 
eady before it was recognized. The fact that the words were separated 
id in ‘ @ made it somewhat easier for him to concentrate attention efie A 
sen } tively. After ten five-minute exercises of this type he showed = 
‘new | marked improvement in regularity of eye-movements and in 
ited fluency in reading. 
ssive In the second type of exercise the words were grouped together 
: in thought units as far as possible. Exercises of this type are « al- 
out- || culated to promote a regular progression of eye-movements, and 
pro- J in addition, the recognition of more than one word at each fixation 
ular 9 Ii, while reading, the subject gave ev vidence of wandering move- ' 
ility | ments, his attention was directed to the s pecific word or words | 7 
cant ™ which were causing difficulty. It should “a added at this’ point a 
rec } that during a period of ten exercises the number of corrections 
} which were necessary gradually decreased. 
| in- In the third type of exercise the words were written one-letter 
the 9 space apart. The subject read aloud five minutes each day while 
bbe, the experimenter noted for evidences of ineffective return sweeps 


Uni- “ and irregular progression of attention. If irregularities were 
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‘ted, suggestions, calculated to correct the difficulty, were of- 
d. At the end of nine weeks marked improvement had been 
le as evidenced by greater fluency and fewer irregularities. 

A second series of exercises was organized as a means of direct- 

r the attention of the subject to important details of words and 
increasing the span of recognition. These exercises took the 

rm of eleven drill books each of which contained ten words or 

irases of approximately the same length for use in short exposure 


cercises. The first exercise of each book appears in Table III. 


he words of Book I were exposed in order. Less than a second 


vas allowed for each exposure. A score of one was allowed for 


h word accurately recognized. The table shows that on the 


rst day the words of the first three books were recognized with- 


TABLE I. RECORD OF THE SHORT EXPOSURE EXERCISES DURING 


A PERIOD OF TWELVE DAYS 


ok |First Exercise of Each} Day |Score Made at Each Exposure} Number of 


Book Exposures 
ion 1 (10 1 
m ‘has 1 (10 1 
mm jbank 1 10 1 
tv. lof wind 1 (6,8 9 
2 8, 8, 6, 6, 10, 10 
3 
4 
vi (the willow buds 4 |7, 6, 6, 6, 6, 8, 7, 10, 10 17 
5 |6,9,9,9, 9, 10, 10 
6 10 
vir jhe said 6 (9,9, 9,9, 10, 10 7 
7 
VIII |pretty soon 7 |9,9, 10, 10 5 
10 
|What is that? 8 |3,3,5, 6,6, 7, 8,9, 9,9, 10, 10 13 
9 10 
x |to her fairy story 9 /2, 2, 4, 6, 7, 7, 8, 9, 8, 10, 10 19 
10 |7,9,9, 10, 9, 10, 10 
11 10 
xt |their little seed boxes 11 {3, 3, 2, 5, 5, 6, 7, 10, 9, 10, 10 15 


12 (8, 10, 10 
13 10 
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out errors. The phrases of Book IV were exposed twice on that 
day. During the first trial six of the ten phrases were accurate] 
recognized. The subject was given his score but no statement was 
made concerning the character of his errors. During the secon 

trial he made a score of 8. On the second day the exposures wer 

continued until the subject had recognized all of the phrases with 
out error twice in succession. On the third day the phrases wer 
exposed again. Inasmuch as the subject made a perfect score 01 
Book IV, Book V was begun. 

The procedure which has just been described was continued 
for twelve days. At the end of that time the tachistoscopic exer 
cises which had been given in the early examination of the subject 
were repeated. The table includes a summary of the errors mad 
by the subject before and after training on the twelve daily exer 
cises. It shows a decrease in the total number of errors from 180 
to 122. There was a decrease in non-recognition from 39 to 32, and 
in wrong recognitions from 141 to 90. An examination of th 
records for each type of material used shows satisfactory improve 
ment in the recognition of isolated letters, three-letter words 
three-letter nonsense syllables, four-letter words, two-word 


TABLE IY. THE NUMBER OF ERRORS IN RECOGNITION BY SUBJECT 
‘A’ IN THE TACHISTOSCOPIC EXPERIMENT BEFORE AND 
AFTER TWELVE SHORT EXPOSURE EXERCISES — 


NuMBER OF Errors | NUMBER OF Errors 


| Berore TRAINING | AFTER TRAINING 
RECOGNITION SERIES |} No. 
| Non- |Wrong| Total! Non- Wrong. Total 

| recog- | Recog- |Recog: Recog-| 

| nition | nition | nition | nition | 
Two-letter words. ... | 18 0 5 5 3 2 5 
Two-letter nonsense syllables.../ 10 1 6 7 + 2 6 
Three-letter words. . . : 10 | 0 10 10 2 2 4 
Three-letter nonsense syllables..| 10 | 4 | 27 31 3 12 15 
Four-letter words. . | 10 | 4 } 15 19 1 1 2 
Four-letter nonsense syllables. | 10 s 30 | 38 13 42 55 
Two-word phrases. .. . ; 10 | 6 | 26 32 0 15 15 
Three-word sentences | 10 | 10 | 15 | 25 | 2] 4 | 16 
TOTAL 39 | 141 | 180 | 32 90 | 122 
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Phrases and three word sentences. For some unexplained reason 
@e subject did less well on four-letter nonsense syllables than in 
@re original test. Inasmuch as the number of errors made in the 
Se cognition of several of the exercises was distinctly above the 
a re made by effective readers in the first test, it was evident 
@hat short exposure exercises could be continued to advantage. 
\fter two months of remedial work, the subject was thoroughly 
fested again to determine progress. In the oral reading test, he 


Made a score of 37.75. This indicates that he made approximately 
& year’s growth between October and December. 

In the Burgess test, he made a score of 50 on the third-grade 
Sale and a score of 34 on the fourth-grade scale. The satisfactory 
fmprovement may be attributed almost wholly to increased rate 
mi reading. 


In the Courtis’ Silent Reading Test the subject answered twenty 
Questions which is twice the number answered in the first test. 
Bie also made a comprehension score of 90 which is equal to the 
Score in the first test. 

lhe December tests showed clearly that the remedial work had 
Besulted in greater fluency and effectiveness in reading. Although 
She subject was not fully up to standard by the middle of the cur- 
Bent school year, he was permitted to resume regular work with the 
fourth-grade class, and he promises at the present time to make 
§ very creditable record. If he had not been subjected to individ- 
Bal diagnosis, it is probable that the real nature of his difficulty 
®ould not have been discovered. Too many pupils fall into the 
Retarded group, and finally drop out of school, because of difficulties 
Which could be removed. In my judgment, each school system of 
Bonsiderable size should establish a center to which unusual cases 
Bould be taken for diagnosis and remedial instruction. 


a 
RORS 
ING 
Total 
; 
| 
| 
} 
‘ 
122 
j 
4 
ay. 


THE NATIONAL INTELLIGENCE TESTS! 
Guy M. 
Burcau of Tests and Measurements, University of Michigan 
HISTORICAL STATEMENT 


Before the war both Professor Yerkes and Professor Term 
approached the General Education Board for the support of 
sort of school survey which would include the measurement of t! 


intelligence of a good-sized group of pupils. The success of th 


Army Alpha Intelligence Examination made it evident that t! 


same general methods would be applicable for such an examina 


tion of intelligence and that there would almost certainly | 
attempts made on the part of various individuals who had ha 


contact with the army methods to adapt these to the examinatio: 
of school children. It was felt that it would be very advantageou 


to the whole movement of mental testing if this adaptation cou! 
be made carefully, systematically, under the auspices of son 
institution or organization with prestige, and by men who woul 


make a serious and expert contribution. The General Educatior 


Board acted favorably upon these suggestions with the provis 


that the National Research Council should take the responsi 
bility for the undertaking and that a group of four or five psycholo 
gists should cooperate in working out the details. A sum of mone 
was appropriated for the work, and Messrs. Haggerty, Terman 
Thorndike, Yerkes, and the speaker were made the members 0! 


the Committee. 


We met first at Washington March 28-29, 1919, again Apri 
29 to May 2, 1919, a third time October 17-18, 1919, and at 


Chicago in December, 1920. A preliminary printing of trial test 
was made in the spring of 1919, and the final completed scal 
were issued in the summer of 1920. Something like 200,01 


copies have been sold to date. The authors’ royalties, I may add 


are turned over to the National Research Council for use 
further studies of tests. 


It is my object in a few minutes to say something about 
the aims of our committee, the methods by which its work wa: 


' Paper read at the meeting of the National Association of Directors of Educatior 
Research, at Atlantic City, March 3, 1921. 
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the criteria we adopted in selecting and arranging the scales, 

nd the results which are being obtained. 

SCOPE 

Our aim was to produce ia a single pamphlet an arrangement 
c ts that could be applied to any child in the elementary 
k-hool who could read well enough to participate in a group exam- 
Snation. In practice this means from the upper half of the third 
through the eighth grade. An examination that serves satisfac- 
forily in the ninth grade or above is almost certain to be too 
Hificult in content or in time limits to suit pupils in the lower 
grades. The National examination has been given with some 
Buccess in the lower half of the third grade and, though the 
Wistribution tends to be “bunched” toward the lower end, yet 


even in that grade it operates well to locate the brighter pupils 
nnd does not give over 2 percent of zero scores. (Six in 363 were 
reported at Jackson, Michigan.) 
RANGE OF DIFFICULTY 
In fitting the individual tests to this proposed scope of grades 
wes there was, necessarily, a problem of selecting material 


that would afford a proper range of difficulty. The criteria 
provisionally adopted in this respect, which were, I believe, 
successfully met with practically every test, were that each test 


uld be so contrived that not more than 10 percent of perfect 
‘res should be made by the average group of eighth-grade 
»ilsand not more than 10 percent of zero scores should be made 
the average group of upper third-grade pupils. I have not 
had time to verify the operation of these criteria with scores 
from various school systems, but I am sure that they are met, or 
early met, under usual conditions of testing. 
PROGRESSIVE DirFIcuLty OF ITEMS 
In order to produce a single examination that could be used 
th in the upper third grade and in the eighth grade and that 
uld avoid too many zero scores in the third grade and too many 
nerfect scores in the eighth grade, it was felt by all members 
f our committee that the several items within each test should 
iry in difficulty and should be arranged approximately in order 
rom least difficult to most difficult. 
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I am aware that there are theoretical arguments agains: 


measuring speed and degree of difficulty at the same time, a: 
that the arrangement just mentioned may be claimed to be 

attempt of that sort.?, Nevertheless, the experience of the Alp! 
Army Test lends support to our position; and it may be argu 
that if the difficulty of the items is properly distributed and 


time-limit is properly set, the arrangement we have made is 


defensible. In any event, while we cannot argue that the ste; 
of our scale are everywhere equal steps (while, for instance 
score of 120 may not be just 20 percent better than a score of 104 
nevertheless, we can argue that the higher the score, the bett 
the intelligence of the examinee, so that, for children of a giver 
grade or of a given age, the total scores in either Scale A or Sca 
B can be regarded as indicating relative orders of merit and hen 
can be taken directly or indirectly as bases for classificatio: 
which is, after all, the final test of usefulness. 


It remains to be said on this point that the actual arrangement 


of the items in the National Intelligence Tests is an empirical on 
it corresponds to the order indicated by actual test results. 


COACHING AND ITS PREVENTION 


At the first meeting of our committee there was much dis 
cussion of the danger of coaching and of the means of meeting 
it. Professor Thorndike, in particular, urged consideratio: 
of this point. He felt that not only would intelligence testing 
become a common feature of public school administration, but 
that it would become also a common feature of business adminis 
tration, e.g., that business men would use mental tests in selecting 
young boys and girls for beginners in their establishments. H: 
argued also that within some five years practically every city oi 
over 25,000 population would establish special classes for gift: 
pupils and that many parents would seek to coach their childre: 
to pass intelligence examinations given for the selection of pupils 
for these classes. On this account, he urged that any intelligen: 
examination that came into general usage ought to be capable o! 
almost unlimited expansion, that it was desirable to include ir 
our examinations only tests the material of which could be s 


* The recently issued Ayres reading test, for instance, is based on the princi; 
that in a rate test all items should be of equal difficulty and rate should be measur 


directly by quantity done within a given time limit. 
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ensive that it would be possible to produce even as many as 
or forty different forms of the test just by drawing material 
ince from the general reservoir of items prearranged for each 
this. even though the actual forms might not be as good as 
| have been produced by selecting the best of the items and 
ning attention to the production of, say, four or five of the 
possible forms. He further argued that another way to 
ourage coaching would be to dump a considerable number of 
ns upon the market at the outset—enough to discourage any 
who tried to anticipate and prepare for the examination. 
rhe other members of our committee felt that the danger 
coaching was much less serious than this proposal would im- 
and there was argument for restriction of our efforts to a 


y forms. The compromise finally effected seems to me to meet 


situation adequately. To begin with, the National Intelli- 
e Tests appear in two independent scales, A and B, either one 


‘ which will serve well for an intelligence examination. Second, 
here is to be published very soon a second form of each scale 


proof is even now in hand). and this will be followed at 


tervals by three more forms, until five forms of each scale are 


‘lable. These five forms, as will be shown later, have actually 
en tried with school children under proper conditions to cancel 
t errors of time-order. Then the forms have been edited, very 
htly, altering an item here and an item there, to bring them to 


it is probably as near an exact equivalence as can be produced 
instruments of measurement of this sort. I shall return to 
is point in just a moment. Third, there has been deposited 


t the offices of the National Research Council at Washington, 


mplete material for five additional forms (that is, 10 in all) for 
ch test in each scale. The material for Forms 6 to 10 was 


rawn by lot from material gathered by the several experimenters, 
ist as was the material for Forms 1 to 5; there is every reason to 
‘pect that without any further trial it will afford a set of tests 


nearly like the first five forms as these are like each other. 
urth, reports have been filed at Washington showing in detail 

methods used by each member of the committee in preparing 
e items of the tests for which he was responsible. This will 
ike it possible, if the need ever arises, to carry these methods 


rward and produce still other forms. 
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As the matter lies, then, our committee has already prepar , 
and tested in actual use five complete forms of each scale, so that 
if we suppose that our five experimentally equalized forms 5 
each scale are in print and that a prospective examinee sho ‘ 
really sit down to prepare himself in advance to cope with : 
form that might be put before him, he would have to drill hims Mist 
on the following: rect 
1. 80 examples in arithmetic | 
2. 100 sentence completions has | 
3. 120 logical selections 1 
4. 200 synonym-antonyms 
5. 45 symbol-digit associations (which conflict with one another) Dir 


6. 110 arithmetical computations 
7. 200 pieces of information 

8. 200 vocabulary items ‘ 
9. 160 analogies 
10. 250 same-or-different comparisons 


one that he has had a liberal education and deserves to “pass’’ t! 

test. 
. Incidentally, it may be noted that the completion of the | | 
: forms necessitated the assembling, testing, and arranging i: . 
proper form and order of difficulty of 3,730 individual items 
which may explain in some measure why five grown men used u rt 
nearly twenty-five thousand dollars worth of time and supplies | 
in producing these scales—about six or seven dollars per item at 
a matter that always seems to puzzle the amateur mental tester a” ; 
and the layman who thinks that a mental test is something that ; 
can be thrown together overnight by any one with a high-scho _ 
education. sh 
EXAMINERS wit] 


+ 
| 
| If he succeeds in preparing these 1,865 items, “I'll say”’ f 


The aim of the committee as originally defined was to produc: 
an intelligence examination, to quote the resolution adopted 
“that can be used by intelligent normal school and college gradu 
ates with a reasonable amount of special training.” As 
matter of fact, the effort was made, however, to render the 
administration of the examination even easier than this statement 
would imply. And experience has shown that almost all class 
room teachers in the elementary school can be brought, within 
brief time, to administer the examination in proper style. O! wa 
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‘ourse, mistakes occur, and {n any large school system there is 
)sually one room from which the results obtained point to some 


error in applying the scales. This is inevitable when any sort 
tandardized procedure is attempted and need not be charged 
avainst the National Intelligence Tests as a fault of design. 
Mistakes of this sort are usually self-revealing and can be cor- 
rected by a second trial with another form of the same scale. 

In Michigan cities where wholesale intelligence examining 


Aik 


has been attempted under the direction of a central administra- 
tive officer, it has been found feasible to utilize the classroom 
teachers as examiners by distributing copies of the Manual of 
Directions in advance, with assignments of reading covering the 


ninistration of the scale to be used, then bringing the teachers 
together for consultation, giving them the test (with shortened 
time-limits), soliciting questions about details, and emphasizing 
certain points (like manner of giving the directions, strict ad- 
herence to the directions, careful timing, etc.) that are most 
likely to be overlooked or misunderstood. 


ToTtTaL DURATION AND TIMING 


A group intelligence examination for use in the elementary 
schools ought, if possible, to be capable of administration within 
single period of forty minutes. Either Scale A or Scale B 
f the National examination can be concluded within that limit. 
In the distribution of time within that limit, our committee 
proceeded on the assumption that a variety of comparatively 
hort tests was psychologically preferable to a limited number, 
say two or three tests, of longer duration. For this reason it was 
ereed that no one test should exceed five minutes and that 
the time should, as a rule, be placed at two or three minutes 
without ever using fractions of a minute (this, of course, to 
ssen errors in timing). The final time limits are, for the five 
tests in Scale A: 5, 4, 3, 2 and 3 minutes, and for the five tests 
in Scale B: 4, 4, 3, 3 and 2 minutes, respectively. 


RESPONSES 


Another principle which is essential to the success of group 
tests of intelligence, at least when working with pupils in the 


f clementary school, is that responses should entail a minimum oi 
writing, should be readily understood by the examinee, and 
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should be unambiguous to the scorer. Some of the group test 
that have recently flooded the market take no account of t! 
principles, which have been found in practice worthy of heed 
which have contributed definitely to the usefulness of the Natic 
Intelligence Tests. 
SCORING 

To make a group test practically usable, the scoring must 
rapid, objective, unambiguous, and simple enough to be un 
taken by any clerk of reasonable intelligence. Here, again, I 
lieve that the National examination, with the possible exceptio1 
the completion test (I say “‘possible’’ exception because even her 
we supply a fairly complete key) is superior to many other gri 
tests of intelligence. Experience shows that no test can 
scored without error, when the task is to handle papers in a who! 
sale fashion. ‘The commonest mistakes made by _ unskilk 
scorers are (1) subtracting wrongs from attempts instead 
wrongs from rights in the R—W form of scoring; (2) giving o1 
half as many or twice too many credits in logical selection; 
failure to multiply loaded scores; (4) adding wrongly in computin: 
the final score; (5) mistakes in counting up scores on a single test 
The experience of Michigan users suggests that, when used i 
quantities, the tests should first be scored by the teachers, an 
that all papers should then be shipped to the office of the chi 
examiner and checked or rescored by clerks trained for this wor 
If the first half of the papers returned by a given teacher are fou: 
to be without error, the remainder may be regarded as sufficient 
certain to be without systematic errors, and the work of the cler! 
may be limited to checking the copying of the figures and th 
addition for the final score. 

The National Intelligence Tests are open to a small elemen 
of criticism in two points. First, the symbol-digit test (Scale A 
Test 5) necessitates multiplying by 0.3. Inaccurate teacher 
are apt to multiply by 3, but this mistake is readily detected o1 
rescoring. Further, the decimal that results in this test, to whi: 
some object, can be avoided by taking the nearest integer for th 
score. Second, the method prescribed for scoring the logica 
selection test when three responses were marked of which two wer 
correct proved bothersome to many teachers. This particula' 
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was formulated, I may say, over my objection (I had charge 
test in question).$ 
[he keys that are provided for use in scoring have, so far as I 
w, met with universal approbation and contributed materially 
the successful use of the examination. 
INSTRUCTIONS 
Unusual pains were taken by our committee with the prepara- 
of the instructions for each test. Special endeavor was 
to obtain simplicity, brevity, concreteness, and clarity, 
emphasis upon clarity. 
When clarity could not be obtained with brevity, there was 
hesitation in sacrificing brevity. Thus, the instructions 
the logical selection test are somewhat lengthy, and those 
r the symbol-digit test still more so, but this length was decided 
pon by actual trials that showed the necessity for every word 
A good example of concreteness is afforded by the directions 
the analogies test, which differ, for instance, from those 
nmonly given to adults in that all reference to the term ‘“‘re- 
tion” or the term “‘proportion” is avoided and dependence is 
{upon concrete directions for conveying the idea of the task 
It is my belief that the National Intelligence Tests can claim 
nusual merit in the formulation of the test instructions, and 
hat the success attained in this regard is another good example 


of the necessity, in the preparation of tests for children, of using 
nothing that will not stand the acid test of experience under 


orking conditions. 

The essence of the instructions for each test is not only 
efinitely brought out in a fore-exercise; it is also printed con- 
isely at the head of the test proper, where it can be consulted 


I need be. 


FoRE-EXERCISES 
Our committee decided at the outset that each test should be 
receded by a suitable fore-exercise that should consume not to 


*I have recently checked over enough papers to prove that the simpler method 
ring that I urged at the outset is perfectly satisfactory and will not appreciably 
the scores or standards for the entire examination, except to reduce two or three 
s the average scores for pupils in the third and fourth grade. The forthcoming 

ipplement to the Manual of Directions for these tests will remove the unnecessarily 
mplicated method of scoring that was originally prescribed for this particular test. 
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exceed one minute and in most cases not to exceed a half min 
that should be so designed as to make still clearer the natur 
the task to be performed, and that should at the same time ser 
to equalize the knowledge with which the examinees undert 
the test in case some of them may have had previous acquainta: 
with the test and others not. 

This feature of the National Intelligence Tests has met wi 
universal approbation and is certainly one of their points of mer 

Another feature of the instructions of the tests is the int 
duction, in the fore-exercise and at the head of the tests the: 
selves, of samples in which the task is set forth and the meth 
of responding is made clear. This feature is, of course, 
peculiar to these tests. Its usefulness is apparent without arg 
ment. 

CHOICE oF TESTS 

At the first meeting consideration was given to practical 
all varieties of tests that had been proposed for use in grou 
examinations. The experience with the army tests was canvass« 
and each member of the committee contributed suggestio 
at length. Various tests were rejected by unanimous consen! 
e. g., memory for digits, the maze test of Beta 1, the counting 
cubes test of Beta 2, etc. 

Agreement was finally reached upon a list of twenty-two test 
that seemed worthy of preliminary trial. These tests were | 
following: 


Printed directions Geometrical construction, 
Disarranged sentences or form combinations 
Arithmetical problems Copying designs 
Information Vocabulary 

Opposites Picture sequence 

Practical judgment Pictorial analogies 
Number series Recognitive pictorial memory 
Analogies Sentence completion 

Series completion Pictorial similarities 
Symbol-digit Computation 

Comparison Logical selection 


Picture completion 


There was discussion, also, of the possibility of developing 


‘ 


an “omnibus” test and of a test that would entail ability 
organize, to carry through material demanding a wider scope | 
attention than necessitated by the “response’”’ type of test illus 
trated in the twenty-two tests just enumerated. 
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THE PRELIMINARY SERIES 


rhese tests were prepared by various members of the com 
e and finally put forth in printed form in eight booklets. 

Verbal A comprised six tests (arithmetical problems, directions, 

‘mation, opposites, practical judgment, and analogies) 

Verbal B comprised five tests (computation, vocabulary, sen 
tence completion, disarranged sentences, and logical selection) 

Von-Verbal A comprised five tests (picture completion, series 

npletion, number comparison, symbol-digit, and form com- 

itions) 

Von-Verbal B comprised five tests (picture absurdities, copy- 

designs, picture sequence, recognitive memory, and pictorial 

ilarities). 

The fore-exercises for each booklet were in these trials as- 
sembled together in four additional booklets—a plan which was 

ter abandoned in favor of the insertion of each fore-exercise 
just in front of its test. 

These four booklets of twenty-one tests were applied to pupils 
in the public schools of Alexandria, Virginia, Richmond, Virginia, 
‘1 the Horace Mann School, New York City, and in three private 

hools in Cleveland, Ohio. The data from these pupils were 
forwarded to Dr. Truman L. Kelley, at Columbia University, 
‘long with teachers’ estimates of intelligence, information con- 
cerning the age and grade standing of each pupil. Dr. Kelley, 
issisted by a staff of clerical workers, subjected the data to very 
elaborate study. In the fall his report was used by the committee 
to supplement its data gained in other ways concerning the merits 
of the several tests. 

Che result, without going into details, was a decision to utilize 
ten of the twenty-one tests that had been tried and to issue these 
ten tests in two batteries of five tests each, either battery as good 
as the other and each composed of an entirely independent array 
of tests. 

The merit of this decision seems to me unquestioned. With 
five forms available soon in either scale (and ten if need be), the 
examiner can repeat a given battery with another form and secure 
virtual identity of tests and equivalence of items and score or 


‘Number series, though in the list of twenty-two tests, was not tried in these 


mrphiets. 
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he can repeat with the other battery and thus get a measuren 
of the intelligence of his group from a different, but equ 
valid angle of fire. 
NON-VERBAL TESTS 

In the preliminary series of tests a definite effort was ma 
as I have indicated, to devise non-verbal tests that would s! 
satisfactory validity as measures of general intelligence—t 
because the committee felt, as most experimenters have {i 
that there is danger of stressing linguistic ability to the poir 
of identifying it with general intelligence. 

The outcome of this attempt was disappointing. Probab 
some of the non-verbal tests that we tried could be further in 
proved,’ but we could not find more than two non-verbal test 


that seemed to justify their retention, and these two have bee: 


used, one in Scale A and one in Scale B. 


Power TEsTsS 


The term ‘“‘power tests” has been applied, perhaps somewha 


loosely, to tests in which there is no time-limit or in which t! 


time-limit is so extended that most examinees go as far throug! 


the series of progressively more difficult items as their abilit 
permits before the signal to stop is given. Many persons f 
that there are certain forms, at least, of test in which speed shou! 
be subordinated to “power,” in the sense just described. In t! 
National Intelligence Tests, Nos. 2 and 3 in both Scale A an 
Scale B are planned to operate as power tests. Examiners shoul! 


not be surprised, therefore, as some have been, to find children 


finishing their work with these tests before time is called. 


EQUALIZATION OF THE FIVE Forms 


I spoke a moment ago about the trial of the five forms of each 
scale and the minor adjustments that have been recently mad: 
to accomplish their practically complete equalization. You may 


be interested to know how these forms came out in this trial. 


These forms were applied to six hundred pupils in grades 


high m1 to high vu, inclusive, in certain schools at Washingtor 
D. C. Without stopping for the scores in the individual tests 


* | still think, for instance, that the form combinations test upon which I exper 
mented, has possibilities that a better form of presentation might bring out. 
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tw 


the total scores for the several forms before any attempt at 
istment was made, were as follows: 


Form 
1 2 3 4 5 
\ 114.9 111.8 112.3 114.4 113.6 
108 .4 109.1 112.0 108 6 109.5 


(hese figures furnish excellent evidence of the reliability of 

method by which the several forms were prepared. The 

adjustments of which I spoke may be assumed to bring 

lorms to a degree of equivalence such that the total score of 
ny form will not, at the outside, differ from the total score of any 
ther form of the same scale by more than 0.5 points, or about 
5 percent. 

RESULTS OBTAINED 


The norms in the Manual.—In the first edition of the Manual 

Directions tentative norms were supplied, based upon results 

m 2,000 pupils in Washington, D. C. and 2,000 pupils in 
Pittsburgh, Pennsylvania. These norms apply to pupils tested 
bout June 1 and tested with Scale B on the day following the test 
with Scale A. These norms were confessedly tentative. 

The results we have obtained in various Michigan cities 
run almost uniformly below those reported in the Manual. On 
the other hand, norms recently reported from East Orange, 
New Jersey, and those reported from certain California cities 
run higher than those reported in the Manual. 

In comparing these results several things have become obvious: 

1. Communities differ decidedly in the general level of intelli- 
gence as revealed by the tests. The results we have on file at the 
Bureau of Testsand Measurements at Ann Arbor leave no doubt 
upon this point. The figures for various Michigan cities, for 
instance, differ consistently from one another, grade compared 
with grade and age compared with age. We have concluded that 
each city must work out its own standards, and that probably 
it will be worth while eventually to work out certain state 
standards. 

2. The norms reported from various communities are defi- 
nitely affected by the composition of the groups under test. Thus, 
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some communities do, some do not, {nclude pupils in speed class 
Some include negro pupils; some exclude them. Some begin 
the lower third grade; others at the fourth grade. Some go on 
through the sixth; others through the eighth. Some have go: 
above these limiting grades to piece out their age standa: 


by including pupils advanced in the grades; others have not 


It follows that age standards (and grade standards, too) will hay 
significance for comparative purposes only when the limitatio: 
of the groups under test are definitely stated and definitely allow: 
for. 

3. The time of year at which the examination is held is a mor 
important factor than seemed at first probable. Two graduat 
students in the mathematics department at Michigan volunteer: 
to derive a formula for correcting scores to a uniform date. Thi 
work has been done, and we shall include the corrective formu 
in the next edition or in a supplementary edition of the Manu 
of Directions. Incidentally, it may be noted that these corr: 


tions are complicated by the varying ages of the groups that ente: 


the schools in September and in February and by the proporti 
of repeaters in various grade groups. The correction for Scale 4 
is 1.36 points per month. In addition, the correction for a clas 
entering in September when compared with one entering i 
February is plus or minus 2.0 points, depending on which way th 
comparison is to be made. 

Deriving mental ages and percentiles—In illustration of th 


point just made—that city school systems will need to derivg,their 


own standards—I may cite the method adopted for this ptrpos 
in Jackson, Michigan. 

The National Intelligence Tests, Scale A, were applied in that 
city to about twenty-five hundred pupils in grades m1 to v! 
inclusive. Pupils aged ten and eleven years in the seventh grads 
were also assembled and tested to perfect the distributions for 
those ages. 

Since most of the information concerning the location ol 
children in the grades is familiar to teachers and supervisors 
in terms of mental age, it was felt worth while to translate the 
scores of the National tests into “Jackson mentalages.”’ This was 
accomplished by regarding the median score of pupils of each ag: 
group as the standard score for the mental age as well as the 
chronological age of the group in question. Thus, all pupils aged 


— 


ht 
gistn 

4 er 
ili 

Bnd! 
| t] 

il 

nt 

} 

mi 


my 


1921 NATIONAL INTELLIGENCE TESTS 29 


ht (over eighth birthday and under ninth birthday) were 
‘tributed in such a way as to locate the median and all the 
er deciles, and this median was regarded as indicating a mental 
of 814 years. The medians for 915, 1014, and 11! years were 
ted similarly and points midway between these medians were 
Len as the scores indicative of mental ages of exactly 9, 10, 
111 years. The amount of overlapping was shown graphically 
the percentile chart, and this chart became directly useful in 
iting pupils of any desired degree of deviation from the 
ndard adopted for a given grade or group. Thus, pupils 
-e drawn off for consideration in connection with special 
.sses and speed classes, for double promotions, etc. The chart 
nercentiles shown as Figure 1 is contributed by Miss Helen 
-vis. one of the recently elected members of this association. 


| 
| 
|_| 
© 10 30 50 90 100 130 14 160 170 
SCORE 


URE 1. PERCENTILE CHART FOR THE NATIONAL INTELLIGENCE 
TEST, SCALE A, FORM I. (DATA FROM JACKSON, MICHIGAN) 


Relation to Binet results.—Another possible method of deter- 
mining the mental age equivalents of the National examination 


* It will be understood that in this chart each of the four age-groups of pupils has 

n reduced to a theoretical 100 pupils. The figures on the base line are the scores 

ed: the figures on the vertical lines are the numbers of pupils in order of excel 

Thus, in the group aged 9 years (median age approximately 9 years, 6 months) 

twentieth pupil in a hundred counting from the poorest pupil scores 28, the fiftieth 

edian) pupil scores 49, the eightieth pupils scores 87, etc. Or, again, 25 percent 
the 8:6 group score as high as the median of the 9: 6 group, etc. 
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scores is, of course, to equate these scores with the results of Bir 
examinations. I am not sure that this method is theoreticall) 
justifiable as the one just described, namely, that of working 
directly to the mental age by reference to the median score. |; 
any event, I do not have available at this moment statistics t 
indicate the equivalence to mental ages in the two systems « 
testing. 

I have some figures of interest that refer to a group of thirty 
two unusual cases from the Jackson, Michigan, groups just mer 
tioned. Twenty-four of these are pupils of inferior intelligen 
under consideration for transfer to an ungraded class or for wit 
holding of promotion. Their I. Q.’s range from 58 to 88, 
tested by the Stanford Binet. Their mental ages, as located | 
the method of determining mental ages for the National test 
previously described are in 12 cases from 1 to 18 months lower 
and in 12 cases from 3 to 22 months higher than the Binet ment 
ages: on the average, the National mental ages are 15 days lower 
so that the net agreement is striking. 

Five of the cases are pupils of superior intelligence, under 
consideration for transfer to a speed room or for extra promotior 
Their I. Q.’s by the Stanford Binet range from 112 to 161. Their 
mental ages by the National tests are 5 and 28 months lower i: 
two cases, and 5, 7, and 8 months higher in the other three cases 
Here the National tests give a lower rating by several months o1 
the average. In view of the fact that these mental ages are menta 
ages as determined by the local results in the city of Jacksor 
where the general scores by ages run lower than those reported i! 
some cities, I am at a loss to account for the seeming discrepancy 
though the instances are too few to be significant. 

Finally, there were in this group three special cases of interest 
The first was a speech defective whose Binet I. Q. was 79, but 
whose National Intelligence rating ran 24 months lower than the 
Binet mental age. The second was a pupil with an I. Q. of 9% 
described as a very poor reader, whose National mental age was 
13 months lower than that assigned by the Binet test. The third 
case was a pupil whose Binet I. Q. was 112, but who is likewis 
characterized as unusually poor in reading. Here, again, th 
National mental age is strikingly discordant, being 38 months 
lower than that assigned by Binet testing.’ 

’ The Binet mental age was 10 years, 8 months. The National score was only 2! 
or roughly that of pupils about 7!4 years old. 


N 
he 1 
De g 
is 
Bort 
evel 
1] 
resu 
| 
| 


2] NATIONAL INTELLIGENCE TESTS 31 
Bir rhese three cases strongly suggest, what would be a priori 
ally zible, that pupils who have special difficulty in reading 
orking ‘fer a decided reduction in mental age rating by a group 


ral examination of the Binet, though it remains possible that 


tics t 
rmer rating may be the more significant in predicting school 


} 
y 
e. Tr i ‘elligence test when compared with their rating by the individual 
a 
ms 


Tess. 


hirt) They suggest, also, that pupils whose scores do not accord 

men- what is anticipated by teachers or who are known to be 

zen Dossessed of some special ability or disability ought probably to 

with. ™pe given individual examinations before any important alteration 

38, $< made in their school status. 

ed | : 

test CONCLUSION 

lower Speaking as a member of the Committee who devised the 

lent National Intelligence Tests, I presume I am influenced by a 

ower leertain amount of unconscious bias in their favor. I have, how- 
ever, sought to speak objectively, to present as directly as I 

under Mecould, some of the facts in the brief history of these tests, the 

tior hims of their makers, the criteria that were observed, and the 


Their | mresults that are being obtained. If I may be permitted, therefore, 
rer it to let my sentiments dictate a final sentence, I should like to say 
-ases | that I feel that the committee has a right to feel a tinge of pride 
hs or in what it has accomplished. 
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THE MEASUREMENT OF LANGUAGE: WHAT IS 
MEASURED AND ITS SIGNIFICANCE! 


ERNEST J. ASHBAUGH 


Director Bureau of Educational Service 


Extension Division, State University of Iowa 


If we include under language all forms of verbal expression 
have to consider not only the most extensive of all the possi 
fields of pedagogical measurement but also the most comp 
We have now progressed far enough in the measurement mo 


ment to recognize the fact that a single measure of any comple: 


quite as likely to conceal as to reveal, and that it is of very litt 


value from the standpoint of diagnosis and remedial work. 

This paper confines itself to the field of written expression 
is realized that many other phases might well be discussed un 
the term “measurement of language,” but a desire to keep 
paper within the probable time allotment, and to speak very di 
nitely rather than in a general way, forces this limitation. 

The various scales for the measurement of composition il] 
trate very well the above statements. The Hillegas Compositi 
Scale, the Thorndike Extension of the Hillegas Scale, and 1 
Nassau County Supplement by Trabue were all designed to me 


ure general merit with the consequent result that the scales ar 
very diflicult to use and the result obtained does not greatly assi 
the child or the teacher in remedying defects and in thus progres: 
ing toward the goal desired. It is to be remembered that the cas 


under discussion is the measurement of language, and what is s 


here and will be said later pertains directly to the use of thes 


scales as measuring instruments. 
Ballou in developing the Harvard-Newton Scale attempt 


at least partially to remedy this defect by constructing separat 


scales for each of the four forms of composition. However, at 
one who has read many children’s themes will instantly rec: 


that children seldom write themes which are wholly narrativ 
descriptive, argumentative, or expository. They have a very dis 
concerting way of mixing two or more forms into one ingloriou: 


whole. 


' A paper presented at the meeting of the National Association of Directors 
Educational Research at Atlantic City, N. J., March 3, 1921. 
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rhe latter scale does attempt to be of assistance to the teacher 
me supervisor in one way in that certain merits and demerits of 
ez the samples in the scale are pointed out. This is certainly a 
> nmendatory feature, but the scoring of compositions for general 
Berit even with these aids is still far from an objective matter 
By ly careful intensive training will develop one to the point where 
he be fairly sure his scoring will have slight deviation from the 
Borm or established value. 

Phe composition scale developed by Breed and Frostic for the 
@xth-grade attempts a narrower range of merit with consequently, 
SB creater homogeneity, but in other respects it adds little or noth- 
®. to the solution of the problem beyond that accomplished by 
th les previously named. 


Willing made a distinct contribution when he provided in his 
sc separating the mechanical elements of capitalization, 
Sunctuation, spelling, and paragraphing from the thought element 
miter having done this. however, he seemed to have felt under the 
Becessity of combining for a single score; and thus, so far as the 
a ‘tself is concerned, it is no more enlightening than it would 
Bave been if he had not separated the two factors in the first place. 
The accuracy with which the score is secured is probably greater; 
But whether the compostion is poor because of mechanical defects 
0 - of weakness in thought elements is completely hidden. 

What has just been said concerning the scales for measuring 
mposition is not to be construed as an argument against their 
A trained person using a scale can doubtless obtain valuable 
1e general merit of compositions written 


r pecaus¢ 


formation concerning tl 
. given class: and this information may be rendered still more 
table when compared with the norm for children of the same 
elsewhere. There is some evidence that scales designed to 
isure general merit may be used rather effectively as a teaching 
vice in much the same way that the various handwriting scales 
ive proved to be valuable. When samples from the scale are 
iced upon the wall of the room and the children are encouraged 


compare their own products with them, they seem to be able 
study these samples and modify the quality of the compositions 
terms of the general qualities considered in the scale. 

What has been said is only designed to point out the nature 


sof the product which is to be measured and the fact that its com- 
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plexity makes the measuring instrument of very little value {; | 
the standpoint of diagnosis and remedial work. 

Any analysis of written expression (which of necessity m: 
the phase of language ability which will be most frequently : 
ured) must recognize at least three groups of factors, 1 

nechanical, grammatical, and rhetorical. Each of these gr 
contains many factors or elements, and only as we separate t 
complexes into simpler and simpler elements will our measurer 
become truly helpful to the teacher and supervisor in the imp: Bi 
ment of the work in the classroom. 
Trabue in his Completion Test Language Scales attempt: | 
devise an instrument “‘for the measurement of ability alon; 
tain lines closely related to language.”’ He states that “E! 
haus who invented the completion test method characteriz 
as ‘a real test of intelligence.’ ’’ Those who have used these s : 
have been more inclined to characterize them as intelligenc: ; 
than as tests of language ability. Using Scale A Trabue repor t 
a correlation with Hillegas Composition of 0.72 for a group o! : 
seventh-grade children, a correlation of 0.85 with the Binet 
for a group of 39 boys, and a correlation of 0.74 with the bi 
tests for a group of 50 boys and girls. Dr. H. A. Greene of! 
University of Iowa, using Trabue’s Scale B and Hillegas com 
sition scores from 132 high-school students in one school, four 
correlation of only 0.38. In another high school the same t 
. gave a correlation of 0.14 for a group of 58 students. vel 
From these data it would seem that native ability rather t . gi 
school training is the determining factor in accomplishment gr 
this test. Possibly the same is true, though to a lesser degre: 
achievement in writing compositions. To the extent that s 
measure intelligence rather than training, their use in the solut 
of the teacher’s problems in language instruction must be negat 
and general rather than positive and specific. 
Greene’s Organization Tests are the result of a definite attem 
to analyze language ability and to present an instrument for | 
measurement of a single phase of this very complex problem 
does not claim to have reduced the problem to the measureme: 
ofasimple element. The ability to take a limited number of id 
presented in jumbled order and to rearrange them in logical or 
is evidently closely related to language ability and is certai 
simpler than the ability involved in writing a composition. 


MEASUREMENT OF LANGUAGE 


Het und a correlation of 0.52 with composition fora group of 


‘eh-school pupils and a correlation of 0.47 with composition 
up of 109 elementary-school pupils. Since his correlations 
erman’s mental age (individual measurements) were 0.41 
0.50 respectively with high- and elementary-s¢ hool pupils, 
les that his test is a better measure of the elements in 
d in language ability than is Trabue’s Completion Test 
recent article in the JoURNAL OF EpUCATIONAL RESEARCH 
‘ted that this test is “mostly an intelligence test’’ but 

| to present any evidence in support of his statement. 
Granted that this test will measure somewhat accurately the 
ability which it is intended to measure, the fact 


language ¢ 
with this test as with many others) that teacher and sup- 


as much in the dark after as before its use 


or remain quite 
erning what should be done to improve this ability. A knowl 


pful unless at the same time 
‘al treatment is at least suggested; and thus far, practically 
known concerning the type of teaching or drill work 


bility and result in greater achieve 


of low scores is not especially hel 


ich will improve language a 
nt on a later test. 

h. in his Grammatical Scale A, limited himself to the factor 
The scale presents two forms of expression, 


f grammatical errors 
ne correct and the other incorrect, and the child is to discriminate 
The task is thus 


tween them, crossing out the incorrect form. 
ry definite. the distinguishing between correct and incorrect 
‘rammatical forms. The sentences are grouped in fours. These 
ps are called steps, and the successive steps are sup yposedly 


f equal increments of difficulty. The scoring is in terms of these 


STatT 


teps of unit value. 

Rather extensive use of the scale has brought out the following 
1. The test shows rather definitely the sentences in which the 
pupil cannot distinguish between correct and incorrect form and 
thus furnishes the teacher with a clue to the instruction needed. 

2. The ability to recognize correct form seems to be more 
losely related to language habits than to knowledge of technical 


rramm 
Yramma 


3. Intergrade differences are so small when 
method of scoring that they become insignificant 


using Starch’s 
and hence the 


re means little. 
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t. On the basis of returns from children in grades seven 


be rather wide differences in the difficul 


eight the re seem to 
individual sentences within the same group See Tabk 
Hence, some other scheme of scoring would probably be of er 


value The facts given in Table I represent a larger number 


ABLE I STARCH’S GR AMMATICAL SCALE 


\CCURACIES (PERCENT 


STEP SENTENCI 
Grade vii Grade vii 

5 1 85.0 94.0 
5 2 93.0 96.0 
5 3 78.0 75.5 
9 1 56.5 74.0 
9 2 74.0 75.0 
9 3 44.0 54.5 
9 4 81.5 84.0 


weil distributed cases than Dr. Starch used in his formation of : 


scale. The sentences from only two steps are given but sev: 
others show similar conditions, namely, that one or two of 
sentences are much more difficult than the others within the s 
Ste 

5. The scale covers a number of types of grammatical er 
but the author has furnished no data showing the basis of select 
of the types and the relative im] ortance of each. 

Charters’ Diagnostic Language Tests, one each for pronoi 
verbs, and miscellaneous, attack the same problem attempted 
Starch, but with a few noteworthy variations. 

1. The material was secured from an extensive survey of 


guage usage among children and was grouped on the basis of | 


three classifications noted above. 
2. Instead of the place of error being pointed out and a cl 


made necessary between a correct and an incorrect form, the c! 


is confronted with a series of sentences each of which he is told 
mark with a cross in case he thinks it correct and to mark out | 
incorrect form and write the correct one in case a correction is ! 
be made. 

3. The test is scored in terms of single unit value for each s 
tence marked correctly. 


| 


w 
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rhe test for pronouns purports to present each of the differ 

ninal uses, and thus covers completely this phase of 
ve ability. The following illustrations show how the test 
int out clearly to the teachers the errors which the children 


Lil 


- recognize as incorrect. The following sentences were pro- 


| correct by more than 75 percent of a large group selected 
om from eighth-grade classes. The sentences are numbered 
appear on Dr. Charters’ Pronoun Test. 
It teaches a person something you may use 
When one lives in town they hear noises. 
\re those them? 
That’s her. 
following sentences were pronounced correct by more than 
ent of the same group of eighth-grade pupils 
Who do you want? 
It was only us. 
He pushed John and I. 
\nnie called to you and I. 
Was that him? 
rhe use of the test with grammar-grade pupils shows clearly 
number of these type errors persist with a very large pro 
f the children who are practically ready to leave the 
and the individual test papers will point out definitely 
teacher the work which needs additional attention. It is 
ible that the sentences should be given a weighted score on 
is of a combination of frequency of social usage and the in 
iracy of eighth-grade pupils, in order that their relative im 
tance may be more forcibly impressed upon those who take 
\ second edition of Charters’ tests provides additional spaces 
the sentences. In this space the child is to write the gram 
al reason for the changes which he makes in correcting the 
tence. This affords a measure of the child’s ability to give the 
rect grammatical rule or principle which applies to the case in 
int. He must not only know the principle but also be able to 
ll it in its proper relationship. 
Kirby has attacked the same problem as Starch and Charters 
He presents correct and incorrect forms for the child to choose be- 
en in the same manner that Starch does. Kirby, however, 
; on the basis of unit value for each sentence. No evaluation 
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has been made for either difficulty or social usage. Here. as 
stated concerning both Starch’s scale and Charters’ test. the 
dren tend to reveal quite accurately the types for which they do 


distinguish correct from incorrect language forms. In this t 


however, no provision has been made to care for the operatior 
chan 


In addition to the check on the language form. Kirby attempt 


to measure the knowledge of grammatical principles involved j 
different manner from that used by Charters. Beside a set 
sentences there are given the different grammatical principles 
volved in this set of sentences. These, however. are arranged 
such a way that a principle is never dire« tly beside the sente: 

hose usage it governs The child is therefore called upon not 
recall from his own memory the grammatical principle which \ 
justify the correction he has made, but to select from a number 
grammatical principles which are before him the one which v 
apply in the particular case. 


\s was shown in the case of the Charters’ tests some of the s 


tences involved language usage which is incorre« tly checked in 


large proportion of cases, while others are seldom missed. |] 


example, it was found with two hundred seventh- and eight 
grade children that the sentence “It is (I) (me)” was missed | 
by 2 percent of the children; “(Mr. Smith) (Mr. Smith he) we 
ahead” by only 1 percent; “There (was) (were many reasons | 
his actions’”’ and “He (don’t) (doesn’t) belong in that group” 

but 7 percent in each case. On the other hand. “He is the m 
who) (whom) you said was injured,” “I (lay) (laid) on the sat 
wo hours yesterday,” “It is a slight to me who (has) (have 

ways been your friend,” and “‘We admire (that) (those) kind 


people” were missed by more than 50 percent of the children o: 


the language side alone. 

On the technical grammar side, the principles governing t! 
following sentences—“‘That book is not (hern) (hers). “(May 
(can) I bring the next story to read?,” “(Leave) (Let) me go wil 


you,’’ and “He divided his money (among) (between) his four 
brothers’’—are the only ones which were corre tly checked | 
practically all of the children. At present it would seem that t! 
tendency in language work in our elementary schools is away fro! 
technigg) grammar with corresponding emphasis upon formati 
of correct language habits. Wherever this attitude is believed 1 
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portant one, the second phase of this test may well be 
ce any test tends to emphasize the type of work which 


1: his teachers to stress the knowledge of grammatical prin- 
he should not measure the children’s lack of knowledge of 
rim iples. 

It is true that ability to recognize incorrect language forms and 
t them does not guarantee that the child will use the 


be 

om 

gap cet the results measured by the test. If the supervisor does 
not 

cip! 

suc 


to 
cor: form in either his oral or written language. Nevertheless 
iti: certain that whenever the child cannot or does not recog 


ne it 


t perfect, that the field has not been completely covered, and 


rect forms when presented to him for discriminating 
ie is not likely to use the correct form when his focus of at 


upon the general thought as is the case in either oral or 

omposition. 

[he purpose of this paper has been accomplished if the follow 

ts have been clearly pointed out. 

First, that the trend in the measurement of language has been 

gross measure of general merit to a specific measure of in- 

idual factors which are included in this general merit. 

second, that different efforts have been made to analyze the 
nd to discover through a study of social usage the phases 
need careful attention. 

rhird, that tests which bring sharply to the attention of 

her and pupils the strength and weakness of each individual 

he phase of language measured are more valuable than those 

ich fail to reveal these situations. 

Fourth, that while a considerable amount of work has been 


is evident that our instruments of measurement are not 


it, therefore, although we are on our way, there remains yet a 
endous amount to be done. 
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INTELLIGENCE TESTS IN CLASSIFYING CHILDR} 


IN THE ELEMENTARY SCHOOL 


CHARLES FORDYCE 
Universily of Nebraska 

For the purpose of determining to what degree the s 
supervisor may through mental tests predict the educati 
career of public school children and thus throw light on 
proper classification, the writer made a study of the results of 
Haggerty Intelligence Examination in comparison with 
school grades and estimates of teachers in the case of a grou 
pupils in the elementary grades at Lincoln, Nebraska. Thi 
were given to 1,078 grade children distributed as follows: prin 
grades (1, 1, and 11) 602 pupils, intermediate grade (v1 
pupils. The Haggerty Intelligence Examination, Delta | 
used in the primary grades and Delta IT in the intermediate gr 
The tests were given by Miss Clara Slade, psychologist in 
Lincoln schools, and the results were tabulated and interpret 
by a class of 67 graduate students in educational measuren 
under the direction of the writer. The test paper of each | 
was corrected by three different students working independent 
The examination was given in the middle of May, 1920. 

All teachers having to do with the instruction of the childr 
gave their ratings of the intelligence of the pupils at the 
the tests were made. The judgment of the teachers was | 
upon class grades made during the year and upon their gene: 
estimate of intelligence. In the case of each pupil the teacl 
rating represents the average of the combined estimates of se\ 
instructors. As will be seen in Table I, the intelligence rank 
given by the teacher, was arranged on a five-point scale: \ 
superior pupils being ranked A, superiors B, average pupils 
inferiors D, and very inferiors E. The rating by the intellig: 
tests was arranged on a similar scale and was based on the I. ¢ 
obtained in the usual way (that is by dividing the mental ag 
the chronological age). Rank 1 in intelligence represents 
students whose I. ©. was 120 or above, rank 2 those with 
1. Q. from 110 to 119, rank 3 those with an I. Q. from 90 to 1 


rank 4 those with an I. Q. from 80 to 89, and rank 5 those wit 


an I. Q. below 80. 
In Table I all entries in the upper left-hand corners ot 


compartments are for the sixth-grade pupils; those in the lower 
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I CORRELATION BETWEEN TEAC HERS’ ESTIMATES AND 
INTELLIGENCE QUOTIENTS 


} D ( B A Totals 
9 49 28 ‘4 0 90 
32 0 0 175 
23 22 ) 0 85 
3 26 42 12 1 4 
11 79 72 21 2 185 
8 53 30 101 
2 32 139 52 9 4 
15 118 263 132 13 541 
l 124 80 4 07 
0 0 14 16 12 42 
1 8 26 47 19 101 
1 12 31 7 
1 10 10 6 
1 5 22 a4 24 76 
1 4 17 14 4 0 
14 108 228 v4 32 476 
60 204 453 233 1078 
46 18¢ 0 139 26 602 
TO 44; P. E.,0 02 
:pper left-hand corners of squares are for sixth-grade puj ils, total 476 
lower right-hand corners of squares are for pupils f grades I, 11, and 111-A, total 602 


n center of squares are for all pupils, total 1,075 


hand corners are for the primary-grade children; while 


ose in the center of the compartments are for all children—i.e. 


entry in the center of each compartment is the sum of the other 
o entries in the same compartment. The detailed tabulation 


ven in Table I shows that for 445 of the pupils or 41 percent, the 


‘ngs of the teachers and those of the intelligence examination 

the same. According to the intelligence rating the teacher 
restimated 383 cases of those of average or inferior capacity. 
is tendency is found to be common and due mainly without 
bt to the fact that many of the average or inferior pupils are 
er aged. On this account, while they are inferior in me ‘ntality 

may be doing only average work in the grade in which they 


— yet they have the maturity of body and the instincts 


nd emotions of children of their own age. They are therefore 
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estimated to a considerable degree by these characteristics. 1 


overestimation is also probably due in part to the fact 
inferior pupils are in many instances attractive, vivacious, 
talkative, always ready to respond to the teacher. These qualit 
usually elicit a good grade. 


Of the average or superior pupils 220 were underestimated | 


the teachers. This may be due partly to the fact that the sup: 7 | 
student is In many instances young for the grade in which | hp - 
placed. He is, therefore, immature in bodily development te: 
in his instinctive and emotional reactions, and is in some deg: It 
judged by these qualities. The underestimation of cap or 
children may also be partly due to the fact that the work is te: 
easy for the superior boy or girl that it does not challenge is 
efforts. Asa result of this he often neglects his assignments, ei: r 
is consequently rated average or low by the teacher 

It is a significant fact that among the 1,078 pupils, 74 per 
of the highest ranks given by the teachers were given to pu This 
who were young for their respective grades. Such pupils in 
higher Intelligence Quotients, and being mentally superior 
their fellows should be expected to do superior work. Ten per th 
of the highest grades were given to the over-aged whose int te 
gence quotients were low. This doubtless was due to the prevalent me 1 


t¢ ndency on the part of teachers to overestimate the older pup : 


Of those ranked average by the teachers 60 percent were of avera, .. TI 
rank as shown by the mental test, thus corroborating the fact $! 
usually found that the average pupil is more correctly judg 3. TI 
and ranked by the teacher than either the superior or the inferior t 


Of the average grades given by the teachers 28 percent were giv: Pp 

to pupils of inferior mentality as shown by the mental test. 0! Bhe g1 
the lowest grades given, 4 percent were given to superior b 
children. This is due no doubt to the tendency to underestim 
the superior or younger pupils or to the fact that these superi 
pupils find the work so easy that they do not put forth an efi 
to do it. One of the greatest values of the mental test is sh 

in this latter case. It discovers the pupil who should be doing 
superior work and leads the teacher in many instances to bring 
about a procedure which will stimulate such students to do better 
work. 


The highest rankings by the teachers were made in the « 
of those of highest intelligence quotients; of the students wit 
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( quotients below 90, 57 percent were given low ranking 
hers, and only 9 percent received above average rank- 
half of one percent were placed by the tea hers in the 
nk. 
edian I. Q. for the entire group was 100 5. The boys 
her in the mental tests than the girls, but the girls were 
rhtly higher by the teachers. The median I. Q. for the 
105. and for the girls 96, but the median rating by the 
as 85 percent for the boys, and 86 percent for the girls. 
ult to explain this discrepancy in the correlation. The 
n between the ratings by the mental test and by the 
was 0.44 with a probable error of 0.02. This « orrelation 
hich as we secured in the case of similar tests in the 


-ade and the high school. 
CONCLUSION 


study in the elementary grades of the Lincoln schools 


it the boys ranked slightly higher in the mental tests 
cirls, but that the girls ranked somewhat higher by the 
estimates. (This is contrary to the results obtained in 
hers College High School where the girls ranked superior, 
the mental tests and the estimates of the teachers.) 


hat the teachers tended to overestimate the inferior 


t and to underestimate the superior. 


t the correlation between the mental tests and the 
estimates was sufficiently high to indicate that it is 
for one to discover by means of mental tests along with 


ies and estimates of teachers such capacities as are essential 


rmining the proper grading and promotion of pupils. 
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t 
estimated to a considerable degree by these ch iracteristics. ] int 
overestimation is also probably due in part to the fact tl by 
inferior pupils are in many instances attractive, vivacious. Pi! 
talkative, always ready to respond to the teacher. These qualiti hest 
usually elicit a good grade Tha 
pepe miu ule WOTK SO easy that they do not put forth an effort 
; to do it. One of the greatest values of the mental test is show: 


in this latter case. It discovers the pupil who should be doing 
superior work and leads the teacher in many instances to brin: 
about a procedure which will stimulate such students to do bet! 
work. 

The highest rankings by the teachers were made in the « 
of those of highest intelligence quotients; of the students wit 
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int sce quotients below 90, 57 percent were given low ranking 

by he teachers, and only 9 percent received above average rank- 

in (ne-half of one percent were plac ed by the teachers in the 
est rant 


Tha madian T © for the entire group was 100 5. The boys 
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THE INTELLIGENCE EXAMINATION FOR 
HIGH-SCHOOL FRESHMEN! 


IrA J. Bricut 
Superintendent of Schools, Leavenworth, Kansas 


We had four general purposes in mind in carrying fi 


he study of the intelligence examination for high-school fres 
rhe first was to determine the extent to which an intelli 
examination can be used at the time a pupil enters high scl 


a means of forecasting his probable success in the various 


school subjects and in order that he may be given intellis 


guidance in his program-making. The second was to pr 
the best possible criteria on which to base the organizati 
class groups. ‘The third was to provide the instructors ir 
high school with a definite statement of the mental abiliti 
the individual members of their respective classes, in order 
they might better adjust their methods of teaching, the su 
matter of their courses, and their requirements to the ne 
the individuals in their respective classes. The fourth pur 
was to determine the adaptability of the Terman Group T\ 
Mental Ability to first-year high-school pupils. 

I am sure no argument is necessary among school m 
prove the statement that up to the present time there has 
very little guidance given to high-school freshmen in their | 
gram-making. On entering the high school the child has 
opportunity to elect certain courses or subjec ts. When advi 
given by advisory committees, class sponsors, or principals, i 
given in most cases without any definite knowledge on the 
of the advisor of the intellectual equipment of the child. 

I know certain principals who always advise those 
complete the eighth grade in their buildings to take Latin 
algebra in the high school regardless of the mental capacit) 
the pupils. A review of the failures by subjects from the Lea 
worth superintendent’s reports for years past shows an excessi 
large number of failures in both Latin and algebra. This i 
doubt due to the fact that a few years ago the distinctive 
ciplinary value of Latin and algebra was unquestioned. It 


‘A paper presented at the meeting of the National Association of Dir 
Educational Research at Atlantic City, N. J., March 3, 1921, 
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it these subjects were worth while for the general 


baht 
ing which they provided regardless of all other con- 


The importance given them by tradition is still 
drawing great numbers into these courses who are 


with discouragement and failure because they lack 
| power to master them. That the loss in the freshmen 
school. due to the lack of an intelligent and scientific 
f guiding pupils in their program-making, is tremendous 
1 be no question. 
scholarship grades used in this study were those given 
igh-school teachers at the end of the first quarter of this 
The intelligence examination was not given until 
ks after the quarterly reports of the teachers were handed 
results of the intelligence examination were not retul ned 
achers until all information from the teachers prompted 
test had been secured. This precaution was taken so that 
wuld be no conscious or unconscious prejudice influencing 
her in the distribution of the scholarship grades or in 
questions concerning certain individuals about whom 
r sought further information. 
tests were all given by the high-school principal after he 
ne over them with the writer and after he had made a 
study of the instruc tions. A stop watch was used so as to 
liscrepancies in the time element of the test. 
erading was all done in the superintendent’s office under 
mediate direction and supervision. The statistical work 
ne by the writer himself. 


he correlation coefficients mentioned in this study have 


e to the Pearson Product-Moment Coefficient of Correla- 
Most of the coefficients were compult d by the long method 


(xy) 
g the formula r= and were checked by ap 


\ Sx" 2 
the shorter method suggested by Dr. Ayres in the April 
r of the JouRNAL OF EDUCATIONAL RESEARCH. 
the correlation tables showing the distribution of tea¢ hers’ 


and intelligence scores, six steps were used for the scholar- 


erades and fourteen for the intelligence scores. In the 


ition tables showing the distribution of scores in ea‘ h of the 
sts of the Terman examination and in the complete test, 
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from ten to twelve steps were used for the individual t 
fourteen for the total score. 

Table I shows the distribution of the freshmen accor 
the teachers’ marks in Latin and according to intelligence 
The median intelligence score of all Latin I students was 
while the median intelligence score for all freshmen was 97 
shows that some selective agency had been operative. 
were, altogether, nine in the freshmen class whose intel 
scores were between 40 and 49 on the Terman scale. Fortw 
only one of these elected Latin. There were also nine fre 
in the next lowest group—i.e., those with intelligence 
between 50 and 59. None of them elected Latin. Out of t 
in the next lower group only two were taking Latin. At th 
end of the distribution the result of selection is quite as 1 
able. In the entire freshmen class there were three whose i 
gence scores were between 170 and 179. All three of then 
taking Latin. There were also three in the 160 to 169 group 
two of the three were taking Latin. In substance the stu 
with the higher intelligence scores elect Latin while those wit! 
lower intelligence scores do not. The selection here shown i 
primarily to two causes. First the general feeling among | 
that Latin is hard. ‘The brighter pupils not having had tr 
in the elementary schools in making their grades are less efi 


by this feeling. Those of lower intelligence have more reasor 


be afraid of a hard subject. The other cause is the fact that 
have given some attention the past few years to the guida: 


pupils in the selection of their courses. We have attempted 


keep out of the Latin classes those who are rated very poo! 
their principals. While the attempt to guide pupils in this n 
has been inadequate, it has been somewhat effective. 

The distribution on the intelligence scale of those ma 


the different grades in Latin indicates in a general way that ' 
correlation between teachers’ marks in Latin and the intellige: 


scores is high. The coefficient of correlation is +0.65. 


The complete distribution shows that 86 percent of all thos 
whose scholarship grades in Latin were above 90, had intellige: 


scores above the 75-percentile, that is they were in the u 
quarter of the whole freshmen distribution on the Terman G: 
Lest. All first-year Latin students whose intelligence scores ¥ 
below the 25-percentile received failing grades in that sub; 
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s). Or stating it conversely, 85 percent 0 
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rTrERMAN GROUP TEST OF MI NTAL ABILITY AND TEACHERS’ 
MARKS IN LATIN 


Teachers’ Marks in Latin I 


Total 
_ 70-74 | 75-79 | 80-84 | 85-89 | 90-94 
a) 1 2 3 
9 2 2 
159 1 2 1 
40-149 | l 2 
139 5 1 1 1 6 
+129 1 7 2 2 12 
119 2 1 3 
109 2 1 1 4 
} 99 1 1 2 
2 3 1 2 
- 79 2 3 1 6 
}- 69 1 1 2 
59 
49 1 1 
Total 11 11 13 14 7 56 
Median 87.5 86.3 125 130 135 121.7 


nly 15 percent of those whose intelligence scores were below the 


lian the group received scholarship grades in Latin above 
f the first-year Latin 
up whose intelligence scores were below the median of the whole 
men group received the lowest passing grade or failed alto- 


er 


I believe therefore it is a safe prediction to say that those 
whose intelligence scores in the Terman Group Test are below 
76, the 25-percentile of the entire freshmen distribution, have 

olutely no chance to make a passing grade in Latin; and that 
those whose intelligence scores are below 97, the median of the 
le group, will do unsatisfactory work in Latin. And I believe 
prediction will hold good for almost any high school, that all 
freshmen whose intelligence rating according to this test places 
them in the lower quarter of the entire freshmen group will most 


likely fail, and that those whose intelligence scores are below the 


median of the freshman group will most likely do unsatisfactory 


Lit 


work. 
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Let us now consider the distribution of English grads 


intelligence scores as shown in Table ITI. 


TABLE Il. TERMAN GROUP TEST AND TEACHERS’ 
MARKS IN ENGLISH 


a! 
Intelli Teachers’ Ratings in English & 
Scores 70-74 75-79 80-84 85-89 90-94 95-100 
9 | ? 
1/0-179 2 2 
160-169 2 
150-159 1 4 2 
140-149 l 2 3 3 
130-139 2 ; 3 5 1 
120-129 1 6 5 2 4 - 
110-119 2 ; 1 3 1 1 1 
100—109 3 2 4 3 6 1 | 
o0~ 99 ; 2? 5 7 2 19 1 
RH RO 2 5 2 3 5 2 ] 
70- 79 2 7 7 6 1 1 
60— 69 3 2 2 1 1 
50— 59 1 1 2 3 2 } 1 
49 1 1 1 
Total 20 27 28 37 6 16 ] 
Median R5 85 90 92 122 143 99 
Table II in a general way indicates that there is some positive | 
relation between teachers’ marks in English and intellig 
scores. The coefficient of correlation is +0.72. 
A more detailed study of the complete distribution 


reveal some facts not precisely indicated in the Table II. It sh 
that 70 percent of all those who received the highest scholars 
grades in English also had intelligence scores in the upper quarter | 
of the entire freshmen distribution. On the basis of Tabl 
the chance is about 1 to 25 that those whose intelligence rating 
places them below the upper quartile will receive the highest | 
(95-100) scholarship grades in English. Moreover, of t! 
receiving failing grades in English, 40 percent were in the lower 
quarter of the intelligence distribution and 70 percent were b« 
the median. 
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( 


49 


+c warrant the statement, I think. that the kind of 


who enter the English classes of the first year ol 
| will likely make. can be rather well determined in 


the Terman Test. 


rrelation table (Table 


- IIT) showing the distribution of 


ies and intelligence scores tells much the same story 


t the correlation is a little lower, the coefficient being 


\BLE II. TERMAN GROUP TEST AND TEACHI RS’ 


MARKS IN ALGEBRA 


Teachers’ Ratings in Algebra 


74 75-79 80-84 
2 2 
1 2 3 
1 1 
2 3 
2 3 2 
$ 4 5 
2 3 
1 
5 1 
2 1 1 
19 33 19 
SO 78 87 


From the more detailed distribution we 


Total 

8589 90-94 95-100 
1 
2 
1 5 6 
1 3 
2 4 12 
2 { 5 13 
1 2 2 
4 4 7 
4 5 16 
3 1 2 19 
3 3 1 20 
9 
2 
1 
27 24 25 147 
99 107 27 98 


find that 88 percent 


e receiving the highest scholarship grades had intelligence 
of the entire intelligence distribu 


ibove the 75-percentile 


\lso, 71 percent of those who fa 
: grade in algebra, had intell 


whole freshmen group. 


classes whose intelligence scores were 


Only 


whole freshmen group received 
‘s. grades in the three upper scholarship groups. 


iled or received the lowest 
igence scores below the median 


30 percent of those in the 


grades in algebra above 


below the median 
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It is obvious that a large number of our first-year hig! 
children are enrolled in algebra who, on account of their low 
gence, can never receive material benefit from it. 1 
illy no chance for those in the lower quarter of th: 


gence distribution to do satisfactory work in algebra, 


prac tics 


little likelihood that those whose intelligence scores ar 
the median will get much value out of the subject. Ton 
it is positively absurd to assume that all high-school pupil 
take the course in algebra. Certainly if all are required 
this course, the subject matter, requirements, and method 
be adapted to the varying mental abilities represente: 
freshmen group 

The distributions in the other academic subject ar 
to those already discussed, and so I shall pass on to the « 
tion of grades in the handicraft subjects. I have inclu 


this group manual training, domestic art, domestic scienc: 


hand drawing, and penmanship. The coefficient of cor 
between ntelligence scores and teachers’ marks in handi 
only +0.36. This is very much lower than the corr 
between the academic grades and intelligence scores 
significance of this fact cannot be escaped. It tells us 

what to do with those of low intelligence who enter t) 
school. It is clear that this group should be encouraged | 
handicraft courses and that their enrollment in Latin and 

especially should be discouraged. 

The high mortality rate in the first year of high scho 
doubt due largely to discouragement which this group is 
to receive, especialy in the academic subjects, under the | 
methods of administering the courses. When teachers 
provided w th an accurate index of the mental abilities « 
pupils and when they have the ability themselves to adapt s 
matter for the benefit of their respective classes, the nun 
failures will be greatly reduced if not entirely eliminated 
have as far as possible organized the classes for first-year 
school pupils who enrolled the second semester on the b 
intelligence scores. The enrollment in the midyear was 


but we were able, for the most part, to put those whose intelli 
scores were above the median of the group into one sectio! 


those whose intelligence scores were below the median of the 


into another section. « From all reports so far received the di 


we 


= 
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juality and quantity of work being done by the two 


e 
arked. 

\\ it is possible the classes in any subject should be 
4 the basis of three standards; those in the upper 
( the intelligence distribution comprising one division, 
1 » the middle 50 percent of the distribution comprising 
i ision, and those in the lower quarter, a third. The 
1 es of the classes comprised of children of superior 
i are almost unlimited. The individuals in these 
5 be held responsible for a much higher quality of 
\ cell as for a much greater quantity of work 

» classes in the lower division the courses should be 

‘ | and these groups held only to standards adapted to 


It is unreasonable to expect the best results in classes 
ere 18 represented such wide variation in mental abilities 


found. 
mind one of the greatest values which come from the 


lligence tests is the stimulative effect of the result on 
‘duals tested, especially those making the higher scores 
nition on the part of the individual that he stands high 
tellectual scale makes an appeal to his pride that is more 
an any other appeal that can be made. 
ceiving the intelligence examination in our freshmen 
he principal of the high sé hool has called into conference 
ose intelligence rating was high but whose s holarship 
were low, pointing out to them the fact that they are not 
ense measuring up to their possibilities and appealing to 
nse of pride to raise the standard of their school work 
proving wonderfully effective and will always prove 
when the principal or teacher is a person of strong 
lity and has the confidence of the student body 
On the other hand, I believe the greatest possible caution 
be exercised in the use of the results of mental! tests In 
place, teachers must learn that the results secured from 
test may, in individual cases, be in error. No teacher 
jump at the conclusion that because a child makes a low 
test he belongs on the low intellectual plane indicated 


ore. Judgment in these cases should be held in abeyance 


test result is verified or disproved | 
reactions of the individual. High scores in an intelligence 


yy other tests and the 
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test are more likely than low scores to be true indications of 
ability. Then, too, an over-evaluation of mental ability 
nearly as serious as an under-evaluation of ability. 

To get a further check on the validity of our test res 
sent out to the teachers concerned a list of the names of th 
who had high intelligence scores and low scholarship grades 


those who had low intelligence scores and high scholarship 


The list was accompanied by a questionnaire in which the t: 
were asked to indicate which of the following factors wer 
responsible for the scholarship grade received by the 


mental ability, attendance, health, indifference, laziness, in: 


effort, attention, energy, affability, application. As an ¢ 
let us take the report on one of these cases. Here is a boy 
intelligence score is 54 but whose scholarship grades are m 
Three of the teachers say that he hasn’t much mental 

one says he has ability. The first three teachers say th 


scholarship grades are due to interest, application, and atte: 


In this case we have the judgment of three teachers agai 
that the test evaluation is about right. 

To summarize the reports of the teachers, 85 percent 
those whose intelligence scores were in the upper quarter 
whole freshmen distribution but whose scholarship graci 


low, were reported by the teachers in this “follow up” qui 


naire as having mental ability. The low grades were acc 
for in this report by lack of application, laziness, inatt: 


poor attendance, and the like. There were 29 in this group 


high intelligence scores and low scholarship grades. Of th 
were credited with high mentality in the “follow up” 1 
There was, then, a material disagreement between the test 
ation and the teachers’ judgment in only four cases of the 45 
upper quarter. 

There were only seven of the 45 in the lower quarter 
intelligence distribution whose scholarship grades were ab: 


that is, in the three higher scholarship groups. In the indi 


reports of the teachers, four of these were rated as havi! 
intelligence. So there were only three of the 45 on whi 
intelligence test evaluation and the evaluation of the t 
differed. 


In short, the Terman Group Test of Mental Ability 


given us a distribution that is most consistent with the tea 


+ 
g 
| 


INTELLIGENCE TESTS Ft IR NINTH GRADE 5 


Ju 
ju mental ability as represented in the s holarship 
gr is represented in the analysis of the individual cases 
ident, then, that the most influential factor in the 
a& vn of scholarship grades is mental ability. The 
g ly that can be given a teacher is an accurate e\ aluation 
0! tal abilities of her pupils. If the classroom work of a 
Pp not measure up to his mental ability, the cause should 
b tely sought and eliminated. It is a tremendous loss to 
t! tal capable of making the highest scholarship grade to 
b. itted to go through school with only average s¢ holarship 
a Phere is a tremendous loss to society as well as to 
t! ‘dual in every case where superior ability is not recognized 
a + given the best possible opportunity tor development 
thoroughly convinced that the efficiency of high-school 


t can be immeasurably increased by the judicious use 

© such intelligence examination as the one used in this 

: It is possible, however, that a shorter examination may 
1 ‘ent for the purpose. To determine whether or not the 

‘ | Group Test could be shortened without impairing the 
results, we found the coeflicient of correlation between each of 
the ten tests and the whole examination and between the first | 
| 
‘ Iv. CORRELATION BETWEEN EACH TEST OF THE TERMAN 
GROUP TEST AND THE WHOLE EXAMINATION | 

Test Correlation 


Coefficient 


0 


1 
0.62 
3 0.82 
4 0.7 
5 0.66 
6 | 0.51 
7 0.75 
8 0.51 
9 0.52 
10 0.75 


ad 
| 
- 
1-5 0.90 
— oF 
‘a “le 


live combined and the whole These correlations are shi 
Table IV in the form of coetticients. Table V shows the 


of the correlation between Tests 1 to 5 and the entire test 


rABLE V. TERMAN GROUP TESTS. CORRELATION BETW!] 
TESTS 1—5 AND TOTAL SCORE 


Scores on Tests 1-5 


Scores on the | ntire Tot 


Test 16- 24— 32- 40— 48- 64—-'72— 80-88 96 
23 |31 |39 |47 |55 (63 |79 187 195 | 103 


170-179 3 2 
160-169 

| 150-159 1 6 i 

140-149 1/;5/4 l 
130-139 1 
120-129 1/4 14 
110-119 11 
100-109 315110) 2 
90— 99 21 
&9 1 12 10 2 
70- 79 4112/8 24 
50- 59 9 
Total 10 17 |17 28 18 13 5 1S! 


These tables make clear, I am sure, that the first five 


may be used as an intelligence examination and that the resu!! 
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\ 


as dependable as they would be if the entire test 
CONCLUSION 


srobable success of first-year high-school pupils 


the 
an be predicted with a reasonably high 


courses C 


.ccuracy at the time they enter high school by the use 


nce tests. 
the intelligence test clarifies the teachers’ problems 


+ the intelligence test offers the best criterion for the 


ion of class groups. 


the intelligence test affords guidance to teachers in 
ition of scholarship grades. 


t the intelligence test, because of its stimulative effect 


lividual, is a powerful agent of motivation 


the results of intelligence tests must be used judi 


- otherwise great harm may be done. 


+ the Terman Group Test is well adapted to high-school 


t the last five tests of the Terman examination may be 


vithout materially affecting the result. 


veneral, that the application of intelligence tests to 
high-school classes is practicable and that unless we 
f them in improving the methods of teaching, in 
courses of study to the needs of the various groups, 


to a greater realization of his possibilities, and in 


in him a desire to measure up to those possibilities, we 


| far short of our opportunities in these things 


eee 
\ Jun 
wil 
we 
Lt 
ag 
or il 
} 
th rll 
if 
‘ 
the in 
fr en i 
pmitted | 
{ il 
il 
Brousil 
lal 
i 
ne, 
‘ 


Editorials 


METRON 


In July, 1920, a new international review called Metr 
launched in Italy under the editorship of Doctor Corrad 
Professor of Statistics in the University of Padua. The 


number appt ared last December. 


This periodical is to apy 


a quarterly and will amount to seven or eight hundred 
per year. It is published by the Industrie Grafiche Ital 
Rovigo (Veneto), Italy at fifty lire. At the rate of exch 
this writing this amounts to $2.04. This is indeed the ¢ 


buy foreign books. 


As a literary venture Melron is unique. It accepts « 
articles in Italian, French, German, or English and print 
in the language in which they are written. Moreover, it 


exclusively or even particularly interested in a special 


knowledge. It is interested rather in the method by which au 


investigate and report on problems, no matter what fie! 


may represent. The one requirement is that papers shall rey 


a statistical treatment. 


methods in statistics. 


Articles are also accepted on alst 
It is in this sense only that Metro) 


be said to have a specialty. 


It is quite evident that the development of statistical i 


has proceeded to the point where it is being erected into 


of knowledge if not into an independent science. This de) 


ment has been going on for some time, and the publicat 


Metron is by no means the only evidence of it. For exam) 
1913 Doctor Hugo Forscher wrote an important book « 
theory of statistics which he entitled, “Statistical Method 
Independent Science.” 


Originally, as the name implies, statistics had som 


to do with the state. 


character. As questions of state came to be nfore and 


closely considered by an ever-increasing number of peopl 


necessity for a method of writing and reporting in a fixed la: 
forced the adoption of numerical terms. Thus a rude pro 


'Forscher, Hugo. Die statische Methode als selbstindige Wissenschaft 


Veit and Company, 1913 


365 pp. 
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It was not necessarily quantitati 
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EDITORIALS 


chich we should now call statistical was built up. The 
a of this method in other fields naturally came about 
a for a rigorous treatment of their phenomena was felt 
ie: 1 have social, economic, vital, and actuarial statistics, 
- biological, psychological, educational, and _ financial 
s Phere is now scarcely a field of knowledge in which 
: is not employed. 
rse. the field in which we are most interested is educa 
1 In it statistical methods have developed with a rapidity 
s quite outstripped the ability of workers in education 
{ orehend it. Yet the time is coming when one can no more 
mpetent student of education without a working knowledge 
’ tictics than one can be a competent actuary oF economist 
uch knowledge. It is therefore altogether appropriate 
cience of statistics should be recognized It is not nearly 
tant that it be called a science as it is that it be accorded 
te position and appreciated as useful for its own sake. 
rst number of Metron is a strong one. The first article 
French) is on “Statistical Method.” It is a brief but well 
ed general article on the subject. Important contribu 
made by such distinguished statistical writers as Gini 
and Edgeworth. As indicating the wide range of topics, 
- note the fact that one article (in English) belongs to the 
of entomology and treats of the length of time which bees 
y from the hive on eat h expedition, while another article 
French) deals with finance, and a third (in Italian) deals with 
racing. This last article by Professor Gini is an interesting 
rison between the success of book-makers at Rome and 
M as indicated by the relation between the actual order of 
' f the horses and the order as expected according to the 
The Roman members of the fraternity were more suc 
| in picking the winners. 
Qur personal interest in a publication such as Metron has 
pages than our 


re 


ps induced us to give it more space in these 
rs will consider appropriate. If we felt justified in devoting 
ore space to it, we might proceed to demolish our imaginary 
nent: in other words, having set up our man of 

proceed to knock him down. We feel quite sure that 
ron is important as a statistical event. As such it should 
interest workers in all fields of knowledge to which statistics 


straw, we 
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apply. We are also sure that because of its polyglot ch 

this publication is important for the graduate student in A: 

universities because of the fine training it will enable 
obtain in the reading of foreign languages. We hope / 
will justify the fondest hopes of its founders. We admir 
enterprise, and we wish them a full measure of success. 

B. R. B 
THE LAW OF THE SINGLE VARIABLE 
In the construction of educational tests there has be 
little formulation of fundamental principles and too much me 
cal application of statistical procedure. When, therefor 
author of a test accompanies the account of its derivation y 
careful statement of the fundamental principles upon whik 
believes tests should be constructed, these principles dk 
the thoughtful consideration of those interested in the con 
tion and use of educational tests. Mrs. May Ayres Burgess i 
monograph entitled Measurement of Silent Reading has formu! 
certain principles which she considers fundamental. Thes: 
epitomized in the title, ““The Law of the Single Variable.” 
The three variables of a pupil’s ability, or more strictly spea 

of his performance, are: (1) quality of performance; (2) le 
difficulty on which it is given; (3) amount of work done wit! 
fixed time, or rate of work. These variables are recogni: 
being complex but they are considered to include all factors \ 
affect a pupil’ ~erformance. 


Performances which depend upon a single variable are i 
trated by the author in the field of athletics. In the high ju 
the variable is the heighth of the bar cleared. It represents | 
dithculty of the performance. The quality and the rate are either 
negligible or constant. In the races the rate is the variable, | 
difficulty and quality are either constant or eliminated. In 
shooting at a mark, quality is the only variable considered. In 
all of these cases the ability of a contestant is measured in terms 
of a single variable. The variable which is used is determined 
the nature of the ability. 

In applying this law of the single variable in the field 
educational measurements the author states that it “‘consist 
distinguishing the possible controlling, varying factors; devising 
means of holding them all constant save one; and measuring that 
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on. Correspondence with the author reveals that this state- 
m not intended to mean what it appears to mean. It 
aj that the law of the single variable was not intended to be 
in’ | as meaning that two of the variables must always be 
ke stant for all pupils while the third was measured. It 
st e interpreted to mean that pupil performances depend 
uy -ee variables, or have three dimensions. In describing a 
p nce these three variables must be re« ognized. If a vari- 
al t constant for a group of pupils its variation must be 
re ‘or each child in the process of testing and when scores 
a rpreted they must be on the basis of a single variable. 
7 mple, if children are to be compared with reference to their 
i work it must be shown that both the quality and the 
d of the work done were the same for all pupils. “What 
t! of the single variable does not permit is the attempt to 
C combinations of the three variables in unknown and 
smounts.”’ 

- obvious that the law of the single variable should not be 
in! ted to mean that when a test is given all pupils should be 
f give a performance which is constant with respect to two 
{ three variables. The characteristics of the abilities which 
‘ n acquire are not restricted to a single variable. When 
. of pupils is performing in a given subject-matter field 
i way which is most natural for each, large individual 


neces are exhibited with respect to these three variables, 
ilarly with respect to rate and quality of york. This is 
even in the case of groups of pupils which ‘iave received 
ne instruction. It is reasonably easy to control the diff 
through the selection of the content of the test. The control 

- rate of work, or the quality of work is not so simple. In 
tain fields it may be shown that one of these factors has little 
upon the performance. For example, in spelling the rate 
work has little effect upon a pupil’s performance, provided the 
allowed is sufficient for all pupils to write the words. In 
fields the rate is an unimportant variable and can be neg- 
This would be true in the case of painting or drawing 

the products are real works of art. In these cases the 

y of the performer depends upon the single variable of quality. 

If a variable is controlled by an arbitrary procedure which 

produces unnatural conditions it appears likely that the ability 
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which functions is modified. or factors are introduced int 
testing procedure that produce the same result upon the | 
ance. For example, if in the case of handwriting all pul 
group were forced to write at a fixed rate, those who wer 
tomed to write more rapidly or more slowly than this rat, 
exhibit abilities different from those which normally fun 
their handwriting. To the extent that this is done the p 
ance secured would not be a true index of the ability 
pupil 

Although the author fails to make it clear in her mon 
the “moral” of the law of the single variable appears to | 
measuring instruments must explicitly recognize the p 
existence of the three variables, or dimensions. of a 
performance. These must be described separately if a 
interpretation of the scores is to be possible. If a vari 
omitted in a pupil’s score it must be shown to have been c. 
in its effect upon the performances of different pupils or 
socially unimportant. This is a principle which has bee 
neglected by many of the makers of educational tests. Ina 
to yielding measurements which are likely to be erron 
interpreted, instruments in whose construction this prin 
violated also lead to implications that are not in agreement 
our educational objectives. Grade norms for a test are gel 
interpreted as standards or goals to be obtained in the resp 
grades. An instrument which measures the pupil’s abi 
terms of the most difficult step reached therefore implies t! 
objectives in that field are to prepare the pupil to do mot 
more difficult things. We should teach pupils to do not 
difficult things but things that are socially useful. Instry; 
which fail to measure the rate of work in fields where it 
important variable are also open to criticism. They do n 
the whole truth about a pupil’s ability. 


Dr. Burgess has rendered a real service in formulating 


principles embodied in the law of the single variable. I; 
fortunate that they are not stated in such a way that misint: 


tation would not be likely. Whether or not future makers of 1 


are prepared to accept the law of the single variable it js | 
that they will feel its incumbent upon them to formulate car 
the principles of measurement upon which the construct; 


their instrument is based. W.S.M 
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Reviews and Abstracts 


E. H. Cameron, Editor 


M Ayres. Measurement of silent reading. New York: Department of 


Russell Sage Foundation, 1920. 163 pp. 


nograph the author describes the derivation of a new type of scale 
lity in silent reading, which is called “Picture Supplement Scale 
Supplementary to this account the author summarizes certain general 
relative to the theory of educational measurements. The Picture 


le is the product of a series of carefully planned experiments. The 


d to the theory of educational measurement are also evidence that 
was given to the fundamental principles on which the construction of 
neasuring instruments should be based. In both cases the author has 
table contribution to our literature in this field. 
spread interest in educational measurements has resulted in the deriva 
eedingly large number of measuring instruments within the past few years 
part the authors of tests have accepted without critical consideration, 
t construction which were originally developed in a single field and have 
without any critical analysis to the field in which they happen to be 
the midst of such work it is refreshing to read the account of the deriva 
Picture Supplement Scale. Statistical tables and the description of an 
e tatistical procedure are conspicuous by absence. The major portion of 
is devoted to a critical consideration of the prin ipl s which should 
nstruction of tests with particular reference in the field of silent reading. 
theme of the part devoted to the theory of educational measurements Is 
the caption “The Law of the Single Variable.” The writer comments 
law editorially elsewhere in this issue of the JouRNAL OF EDUCATIONAI 
| for that reason only brief reference will be made to it here 
tion of the “Picture Supplement Scale” is to measure the amount of 
rial which the pupil can read within a given time. In respect to the type 
vhich is measured the author claims that it is “a test of careful reading.” 
msists of twenty exercises equal in difficulty and printed on one side of a 
t. Each exercise consists of a short paragraph and a picture. The para 
ly descriptive of the picture and in part consists of directions for drawing 
ntary picture. The test of the pupil’s reading of the paragraph is the 
the supplementary picture. The pupils are allowed five minutes to do as 
ey can. Directions for scoring are liberal. Any sort of a drawing which 
t the pupil followed instruction is to be counted as correct. The scale is 
| for use in grades m1 to VIII. 
rivation of this scale the author identified twenty-five factors which 
1 pupil’s performance in silent reading. An attempt was made either to 
r to control twenty-four of these factors, leaving only the amount read as 
factor. The ability to draw the pictures required (difficulty of action 
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inced as being held constant but one naturally quest 
f the drawings to be made are very simple. For exampl 
Hat k« to cr 


) L to cross out a portion of a picture. In other exercises ; 
: x. In Exercise 4 the pupil is to draw a picture of t} 


. truct ire given to the pupil concerning the quality of the drav 


pect to find that some pupils would attempt car: 
ike very hasty sketches 


rhe instructions for administering th 


1¢ tests are meager and do not 


exact explanations to be given to the pupils. The pupils are given no 


exe! t iint them with the nature of the tests. One would, ther 
on of the tests to lack objectivity. One is also inclined to 
ugh the directions state that any sort « 
that a pupil has followed instructions is to be accepted as corr 
om e bureau which is distributing the Picture Supplement Scak 
n iry to issue detailed directions for scoring the exercises 
rhe pupil's score is the number of exercises done corre: tly in the 


illowed for the test Che author considers that both the difficulty of 


‘d are kept constant. The uniform ty of 


ses for which the same percent of correct re sponse 
ty of the reading is considered to be kept constant 


pupil given credit for only those exercises which he answers correct] 


hor the score is a measure of the amount of reading 


n five minutes or, in other words, his rate of reading 
[t is likely that the number of exercises done correc tly furnishes the 
description of a pupil’s performance on this test. However, t 
‘sure of his rate of work. It is a combination of rate of reading and 
lation Is not the same for all pupils. For example, a 
s ten exercises with 100 percent accuracy. Another 


make a score of 10 by doing twenty exercises with 50 percent accuracy. 17 


ne for these two pupils. Neither have they read wit 


Che pupil's score in terms of the number of exercises done correctly is t 


lated into a score on a percent scale. This scale is different for the differ 

For example, if a pupil is in the fourth grade ten paragraphs done correctly 1 
re of lf he is in the seventh grade his score is 50. No real ac 

tion and it will doubtless be confusing to many. 

lhe reliability of the scale was studied by having it given twice to a 

groups of pupils. The reliability of coefficients calculated for these gro 


mat those tor 


. number of other reading tests. 

upplement Scale represents an ingenious and serious att 
iment for the measurement of silent reading ability. It 
place among our best reading tests. If the present form were supplemente 
detailed directions for administering it and by the addition of a few pr 
would be well adapted for general use. particularly in the lower ¢ 
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woop P. The history of education New York: Houghton Mifflin 


1920. 849 pp 


woop P. Readings in the history o education. New York: Houghton 


ny, 1920. 684 pp 


5 r Cubberl y has, thro iwh the ex eedingly useful character of his educa 
following who will welcome the public ation of these 
ntributions to the history of education Professor Cubbe rley 1s 


+ attained to an almost unique position in the e jucational world 
vears he has been directly connected with university work he has 
t touch with the developments of educational administration and, 
writings and his work in various cities as an educationa: surveyor, 


administrators 


ited powerfully to the ge neral acceptance by 


cational administration. Having retained this intimate and con 


ation over so long a period of years one expects to hn 


toward edu 
f the history of education modified by this point of view. 


-alized. as he himself points out in his preface 


f iblic school sys 1S see the 
ally irveying the WOrk OF public school sy tems not to e the 

hools 


It is difficult 


tion which always exists between a Civ ilization and its type of s« 

f educational theory, as such, is forced to the background while 
n of the actual efforts in education is correspondingly emphasized 
ve to the rather general realization of the indefiniteness in the relation 


nal theory and the solution of present educational problems which 


+ the last few years the sharp decline in popularity of the traditional 


At any rate, we have in these two books the 


ry of education 
a man who is primarily interested in the effort of modern society 
lf tively through improvement in hool organization, to pre 


ng itsell 
vhich would treat education as a phase of the ge neral development of 


Phat he has accomplished this in a way whic h will win the hearty ap 


United States almost goes without saying 


‘nclined to be over-critical, he might question the wisdom of including 


in any well-written survey of history 


school men of the 


s of the text so much that is found 
ether the book could not easily have been made much smaller 
1s a presentation of the development of educa 


nd myself continually asking this question; and yet, when I specifi 
i to cut out topics that primarily belonged to general history I hesitated, 


t in spite of the greatest efforts of teac hers of history, much of this infor 
ance of the 


t be so poorly maste red by the student that the real signifi 
s would be lost without this other presentation. 50 I have finally 
lpoint Mr. Cubberley 


} 
wh 


ve reduced to any degree 


criticism believing that from the teaching stan 


nclusion of this material 


Mr. Cubberley’s texts are unusually readable. 
yredict that many schoolmen will, from the motive of 


These volumes are no 


to the rule. I f 
ne, read the text from cover to cover. 
ion have been carefully selected and are so 


M Readings in the History of Educatt 
as to follow the chapter arrangement of the text In this way much that 


4 erwise have had to be included in the presentation is immé¢ diately available 


nd volume. So well has this collection of readings been compiled that none 
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the history of education would feel much need for acce 


not found within the covers of this book. 

Professor Cubberley follows the « ustomary organization of material 
texts now generally used but never loses sight of the point of view stated in 1 
of the volume “Educational practise and progress considered as a phase in t 


ment and spread of western civilization.” He divides his study of western 
into four parts: “The Ancient World,” “‘The Mediaeval World,” “The Tra 
Mediaeval to Modern Attitudes,” and “Modern Times.” He, however, 
fully the vastly greater significance to the student of education of the mo 
and devotes approximately half the book to part 4. Students of history W 
question the propriety of dismissing the educational « ontribution of Greece 
early Christianity with a paltry hundred pages. Possil ly one might quer 


omplete omission might not be justifed with better grace than the cursor 

hich, frequently, is all that is included. I, personally, however believe t 
tribution of emphasis is well justified Unless prerequisites are insisted up 
ol n produces with many readers distorted perspective, while cursor 
suggests the influence upon modern conditions which is all for whi h the 


Certainly he is justified in assuming that the student or reader is not « 


ignorant of ancient and mediaeval history. Therefore he is equally justific 


t volume the chief purpose is to show the vital connections |} 
gnificant development of the civilization of Greece and Rome and the ¢ 
conditions of thos+ ages, and that in turn Christianity not only furnishe 
ich could serve as a foundation for modern civilizatior 
1 fundamentals both of civilization and education without wi 
of reconstruction would have been far more complex and difficult. Cubber 
ment of the significance to modern education of the revival of learning, of t 


tant revolts, of the development of scientific inquiry and of the scientific met 


contorming to that usually found is clear and convinc ing and dramatically 
average reade r 


hool superintendents and principals will be especially impressed 


ions centering around the abolition of privilege, the rise of democra 
theory of education, and the struggle as a result of which the state tak: 
ool 

Those who have read his Public Education in the United States will find 1 
is duplication of the earlier work. This is probably inevitable. Professor | 
could not afford to omit important and pertinent topics merely because he |} 
discussed them in another volume. At the same time. those who read all of | 
may well be pardoned if they regret that he did not see his way clear to var 
presentation of these topics. The same feeling has occurred to me more th 
reading Public Education in the United States and comparing it with his Pu 
Administration. Topics in school administration although legitimately 
sarily found both in a history of education that professes to include the « 
movements that have had their best exemplification in the United States 
special treatment of education in the United States surely can be devel 
somewhat different angles if they frankly make their appeal to the sam 
readers. However, this is really a very minor criticism. Most of us will pr 
reading these topics and 


those who use only one of the volumes receive th: 
view in each case that the author is earnestly desiring to impart. 
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t r I ll r n 

rm f the rley r t 

Hes the older ent n ind 

} t our il org thr our 

: r adjusted to the conditions under which they live and which they 

| 
‘ 

these two volumes to teachers of the history of education in our de- 

| jycation in universities and to teachers of the history of education in 

, ‘ols as handbooks admirably designed to give a student of this subject a 

sew towards education which is eminently sane and constructive. Too many 

t : even administrators fail to realize the cause of many of our present 

problems. Studies of this character develop the power to see the relation- 

seen a given problem and the given so ial, political, and economi conditions. 


oment, therefore, a course in the history of education from this point of view 
ehile for all students of education to take and I would not be at all surprised 
. result of the publication of these volumes a very real increase in the in- 
the history of education as a vital division in our departments of education, 
C. E, CHADSEY 


of [linois 


ey, E. P. The history of education. Boston: Houghton Mifflin Company, 


192 S49 pp. 


s preface, Professor Cubberley tells us that he has “not tried to prepare 
snother history of educational theories,” and that he has “omitted reference to many 
ts and reformers.” His dismissal of Plato and Aristotle in a few sentences, and 
thout giving even the briefest sketch of their schemes of educ ation, is equivalent to 
n of reference. The reader is told that Quintilian was “the foremost Roman 


sriter on educational practice,” and that his Institutes of Oratory gives “‘a detailed 
explanation of the old Roman theory of education at its best,” but is left to wonder 
he old Roman theory of education at its best’’ was like. ‘For such omission I 

ve no apology to make.” Jpse dixit. Very well! This review, then, to borrow the 
suthor’s phraseology, will be “less concerned . . . . with the educational and philo- 
| theories advanced by thinkers . . . . than with what was actually done.” 


r. it should be observed that a well-balanced History of Education, or History 
tion. should contain liberal reference to “theories.” 
ne reads the text he is conscious of passing through many areas of condensa- 
rarefaction. The reviewer questions the advisability of subjecting the casual 
r r. or immature student, for whom the work is designed, to so many changes of 


preface, and the book itself, suggest that it may be more profitable to search 


rrors in the statement of “facts,” than in the presentation and evaluation of 
And their “‘name is legion.” 

ve 7, in the context having reference to the ope ning of the “Christian era,” 

e statement that “Greek was forgotten.” The fact 1s, Greek was not for- 

that time. The following page contains the information that “the study of 

i Hebrew” was “revived” during the “Italian Revival,” and on page 95, in a 


7 | 
| | 
| 
| 
f 
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section devoted to the 


“Rejection of pagan learning in the West,” (four 
centunes 


we read that “the Greek language was forgotten, and was not known ams ¥ 
ie in the West for nearly a thousand years.” 
the statement (page 424) that “Greek 


This incorrect impression is enha; 
was restored to the western wor i 


during the period embraced by the years 1333-1433. The study of Greel 


unknown in Sicily and Italy in the twelfth century, and the “study of | : 4 
Hebrew” occupied a fairly prominent place in the thirteenth century Johr » 
Raymond Lull, Siger dé Brabant, Guillaume Bernardi di Gaillac, and Ar 3 
Villaneuve knew Greek and Hebrew. The Dominicans made provision for P 
in these languages in the t} rteenth century, and the Council of Vienne 131 e 
that the Eastern t e taught at Rome, Bologna, Salam inca, Paris 
It is incorrect to say that iring the Middle Ages none could read . 
The tudy of Gree continued, without interruption, throughout the M . 
A legend that nger obtains among those who have done intensiy r 
in the f ‘ eval literature tinued In such expressions as the fol] be ; 
long intellectual night of the M idle Ages the long intellectual nicht of the + { 
per the long period of barbarism and general ignorance,” and “the |. 
of int tual stagnation.” 
rue enough, during the seventh and eighth centuries, Many of the pr hs 
woef rant {many have been “woefully ignorant” ever since that 
influent few, however, were not so gnorant. And one must take cum » rin 
the statement that “the Latin writ ngs of the time contain many ina 
corr reveal the low standard of learning even among the better a 
or the cr I cl ws Phe tandard of the educated cle rgyv was high: it 
on! mong the uneducated clergy Would it be proper to conclude t] ; 
standard of learning” obtains anx ng present-day writers, of colleg texts ever 
the riting contain many inaccuracies and corruptions’? Cert 
of histor | ld be ol €rved in our evaluations and interpretati: 
The cor lusior 263) that “Mediaeval education prepar 
world t for the world men live in here” is too sweeping Only tl 
religic ducation of the period “prepared for the world tocome.”’ Secular 
whether r tl bles and the well-to-do, or for those of low degree,” d Vl 
for the world men live in here “The world of the mediaeval monk and thx 
tic” (p. 279 not the whole world of the Middle Ages, by any means. 
At one point (p. 155) we read that theology was “the one professional st 
whole midd ge period,” and at others pp. 199, 211) that medicine and ‘ 
prof il s ects” also 
For the nvenience of the reader of this review hve consecutive senten 
occur under the title “Results of their work” (the work of the Schoolmen 
ie. are here reproduced The work of the S hoolmen was to organize and pr t iB 
ar systematic and dogmatic form the t ichings of the Church. This they did es 
Je well, and the res it wasa thoroughgoing organization of Theology as a teaching Mon 
They lid little to extend knowledge, and nothing at all to apply it to the pr 
4 nature and man. Their work was abstract and philosophic al instead, dealir n 
with theol 


logical questions 


The purpose was to lay down principles, and t 
training in analysis. 


comparison, classification, and deduction which would 
learned and subtle de fenders of the faith of the Church.” Evidently the author wishes 
to give the impression that the Sx hoolmen dealt “‘w holly with theological quest 


ee 


‘aye 
No 
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r from the truth! Scholasticism was interested in every phase of 
Only part of its work was that of “reorganizing and systematiz 


be furthe 

activity 

r striking area of condensation contains the following, in two sentences 
translation (New Testament, 1522) virtually fixed the char- 


» German language. Calvin’s Jnstttutes of Christianity (French edition, 


; ‘1 Sxed the character of the French language, and Tyndale’s translation 
0! Ni Testament (1526 _. fixed the character of the English tongue ” 
c .v the least! And these statements occur without qualification 
ve (258) we find that Magellan “circumna\ gated the globe” in 1519-21, 
5 , (424) that his “ships rounded the world” in 1515-15 \ similar dis 
c irs in connection with the dates of publication of Rousseau’s 5 iad 
( Emile In 1752 appeared both his Soctal Contract, and Emil +83). 
] tes that they were published in 1762. In another instance (p. 525) the 
: In 1799 . . . . Jefferson tried unsuccessfully to secure the passage 
( rehensive bill _ for the organization of a complete system of public 
f - Virginia.” If this is a typographical error for 1779 it should have been 
. footnote to page 526, and in the Readings, page 427. Again, an incorrect 
ht be gained from the statement (p. 524) that “New York in 1787 
trative organization known as the University of the State of New 
t th the author’s Public Education in the United States at hand, the rea ler 
r here (p 258) he learns that the Unive rsity of the State of New York 
w liched in 1784, and organized in its permanent form in 1787.” 
r quotes, without comment, on page 369, the following, from Draper 
h schools in the province (of New York) from 1700 down to the time 
yn of Independence were maintained by a great re ligious society 
ty for the Propagation of the Gospel in Foreign Part \s a matter 
ry few of the schools in New York during this period were maintained by 
G. And it is equally incorrect to say (p. 521) that ° In the Middle Colonies, 
t narochial-school con eption of education was the prev tiling type, the school 
ret ! under church control until after the foundation of our national government.” 
M ft hools of the Middle Colonies, in the eighteenth century, were private 
ls and not “under church control.” Again, a faulty implication 1s 
ned in the assertion (p. 520) that “By the close of the « olonial period the new 
Academy (p. 463), with its more practi il studies, had begun to supersede 
t Latin grammar school.” Schools offering these ‘more pract il studies” far 
red those of the Latin Grammar School type long before “the close of the 
eriod.”” 
verage instructor, as well as the average student, accepts the printe d word 
re is observed, to point out certain errors, the text is usable. Its scheme of organi- 
good. It supplements admirably the work that has been done by Graves and 
Manr 
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I tain news items regarding researcl | t 
r relera n nore t } 
sw ll be printe r the 
‘ ¢ 
\ rrespon ice conce! b irtmer \ 


\ letter from Henry D. Rinsland, Director of Department of Re 
Guidance, Ardmore (Oklahoma) City Schools, gives the , 
Educational information concerning educational research activities in 
Research in \t the invitation of County Superintendent Mrs. M. | 
Oklahoma Rinsland, a survey is being made of the public schools of : 
County. This is the first county survey in Oklahoma. The: en 
of the survey staff are from the University of Oklahoma and two of the state al 
schools. The survey has been organized in seven divisions as follows 
Division 1 nization and Administration 
Division 2. Buildings and Equipment 
Division 3. Attendance and Enrollment. 
Division 4. Instruction, Course of Study 


Divi 
Divis 


Divis 


Mr. Ris land 


schools of 


supervising 


Bureau of 
Service Un 
of Kansas 


Lawrence 
ippear 
schools i 


n 5. Teacher Status 
yn 6. Tests and Measurements 
n 7. Finance 


also reports the extensive use of intelligence tests in t 


\rdmore under his supervision. He has also spent some time in H 4 
the giving of educational and intelligence tests in that city. LR 
Siler 
University of Kansas, Dean F. J. Kelly of the School of Educat 
ceived the idea of a bureau whose function would be pr 
School service rather than research. At the present this Bureau 
iversity Service is under the direction of F. P. O’Brien. A recé 
from Professor O’Brien gives the following information « 
the work of this bureau. 
rhe Bureau has directed a survey of the curriculum and building probe 
stem. The report was completed early in January and 
printed fort rhe Bureau is at the present time cooperatin th the 
ties of the first, second, and third classes, in making a study of the 
ition in the schools of the State. The report is nearing completior 
tributed to the school systems in this State so that it will be in t 


districts in 


of distress d 


> time for the re-employment of teachers for the coming year. The 


cting a survey of the school situation as it is found in several of t 
an adjoining county of Kansas. Some of these districts are in a 
to the fact that several small high schools are maintain¢ 
be adequate. The financial burden of the present arrangement is 
districts. They made an appeal thru the County Superintende: 


J 
¢ “ 
M roe, Unive! y of ili rbana, illinoi 
is also ce 
one ou 
for certain to this 


these subjects should be heard tron It takes something more than a 
er to devise acceptable instruments ol this sort. We hope Protessor 
I vations to a successful conclusion 
es r receipt of a “Bulletin on Silent Reading” from Superi tendent F. 5. 
Camp of Stamford, Connecticut. It is addressed to his principa and 
‘ na af the hect we have seer Roth silent and oral 
4 Bulletin on eachers It is one of the est we ave seen. ot l t and orai ; 
Silent Reading reading are divided into intensive and extensive activities, each of 


h is defined and illustrated 


r 
t intensive reaal is largely stud Silent extensive reading rea l 
a jual interests ()ral intensive readl includes 1 e early 
tional mechanics of re ind in the upper grades the study of 
r » st readll 1. €., reaadir in wl i the 
e na are ar ) on p ive 
rs er e per reads to a gr listeners 
these lour typt re t ted 
e t er n ea i the eight et ir it 
r t r e, so far as the pr i ma t 
‘ page 70 The entries indicate er tent en 
ce 
tne | i i} 
et ve et 
eacter ‘ r t ia 
, espe t r t r ce 


) us r 
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» thines senarat Rot the rete ai of 
ant in all these ft t or for 
en the é ts inv ved in § ] | 
r re! l¢ tence se to be open to the ¢ t t tt e not 
t rd vocabulary and standard syntax Unfortunate lections 
t standardized though we are working on that probit 
t thin the bounds of reason that a Latin test 1 prof r of Latin 
edence over every such test now available. We hav ought in the 
tn he i dee d hicher educational institutions 
improvement. 
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SUGGESTED TIME VALUES, TEN REPRESENTING THE BA 
READING 


pe f Grades 


Silent intensive reading 2 1 a . a 0 0 
Silent extensive reading 0 l 2 3 5 6 7 
Oral intensive reading 7 6 5 t 3 2 2 
Oral extensive reading 2 2 2 
* Large time values here in grades where the child must ears from the printed text; | 
onsider as falling under “reading exercises” in the ordinary sense, no ratio numbers are a 


Schoolmen’s Week at the University of Pennsylvania was held Thursday 


and Saturday, April 7-9. There were 1,885 persons re 


Schoolmen’s Week while 8,565 attendances were noted in 31 progra 


University of number of registrations and attendances were both la 
Pennsylvania in any preceding year, the previous highest records, 
3,802 respectively, having been made in 1920. 

Among the speakers from outside the state were, J. H. Kirkland, | 
Vanderbilt University, Nashville, Tennessee; Frank Aydelotte, President 
Swarthmore College, now Professor of English, Massachusetts Institute of T¢ 
John W. Withers, former Superintendent of Schools of St. Louis, now Dean 
of Education, New York University; Walter S. Dearborn and John M 
Harvard University; William S. Gray, School of Education. University 
Professor N. L. Engelhardt, Otis W. Caldwell, and Miss Fannie Dunn. | 
University; and C. J. Galpin, Specialist in Farm Life Studies, Department oi 
ture, Washington, D. C 


Social and recreational features took the form of visitation of the cam 


buildings, including the museum and library. The guests of the university wer 


luncheons on Thursday, Friday, and Saturday at noon, and on the evenings of 
day and Friday 


tendered | 


On Saturday over 650 luncheons were served, a recept 
y the Acting Provost, Doctor Josiah H. Penniman after the Friday « 


program. Several hundred accepted the invitation of the Athletic Ass 
witness the baseball gan 


between Pennsylvania and Swarthmore, Saturda) 
On Thursday night two } 


stage seats at the Academy of Music and the Metropolitan Opera House 


American Legion gave a program of unusual interest for the purpose of stimulati: 


spirit of Americanisn 


It was generally considered the most successful meeting yet held. TI 


was due in large part to the faithful and cordial cooperation of the memb« 


General Committee, the Reception and Registration Committees, and the 
Committeemen, and to the support given by State Superintendent Thomas | 
and City Superintendent Edwin E. Broome and their assistants. The executi 


iundred superintendents and supervising principals were 
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4 e as in previous years—Harlan Updegraff, Chairman, General Committee; 
, i nes in charge of Secondary School Conferences and LeRoy A. King, 
General Committee. 


Fouivalence of Forms I and II of the Burgess Picture Supplement Scale for Measuring 
Silent Reading Ability 
ifter the beginning of the present semester, the Burgess Picture Supplement 
r Measuring Silent Reading Ability, Form I, was applied to about 1,000 
| eighth-grade pupils with the following results: 


Class No. Tested Paragraphs Credit (Median) 
(Median) 


| 
| 
| 


VII-B 238 7 35 
VII-A 263 8 38 
VIII-B 242 9 41 
VIII-A 219 9 38 


[t was expected that later in the semester Form II of the same scale would be 

ed for the purpose of securing a measure of the progress that is being made under 
xis of instruction. 

in order to get an idea of the practice effect of taking the test once, a group of 

sixth-grade pupils was given Form I on April seventh. Under exactly the same 

ns so far as it is possible to secure such a situation the same group was given 

rm II on the following day. The records of all pupils not present at both tests were 


\s in the case of the seventh and eighth grades, this group fell below standard, 
securing a median score of only 43. But to our surprise the median on the following 
lay with Form II jumped to 64. Forty individuals made gains totaling 198 paragraphs, 
four neither lost nor gained and two lost a total of 2 paragraphs. 

ese gains were so notable that it seemed as if they could not be attributed en- 

tirely to practice effect. Therefore another group of 45 high sixth-grade pupils was 

selected and given Form II, followed on the second day thereafter by Form I. Instead 

provement appearing as the result of experience there was manifested a decided 

ss. The median score dropped from 55 to 49. Twelve pupils gained twenty para- 

graphs. Nine neither gained nor lost and twenty lost a total of 55 paragraphs. This 

helped to conform our suspicions that the gains of group one were partly due to a 
difference in difficulty of the two forms and not altogether to practice effect. 

lo further satisfy ourselves as to whether or not Form II is as difficult as Form I, 
a third group of high sixth-grade pupils was given Form II and after a brief rest, Form 
I. The general results agreed with the facts already noted. This class scored 58 on 

rm II and after a few minutes rest fell to a score of 49 on Form I. Seven pupils 
le a total gain of 10 paragraphs, over the record made on Form II, seven pupils 


made the same score and eighteen pupils lost a total of 43 paragraphs. 
Our brief experience with this test may be summarized thus: (1) Form I appears 
onsiderably more difficult than Form II; (2) inasmuch as all of our Form I 
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HT 
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lhe Illinois Examination 

thar ™ eo and re to ¢ 
ee r i tea er hy interest in re 
I Lu t 1 the Princet ty schools 

1 é ter et tively engaged in wor vith this batter 

test r { tv s e the sixteenth of last Au st Che exa 
ce tv iy ed ee hundred eighty-five teachers ind about e t 
hundred boys and girls. It impossible to give any account of the Pri 


school thout discussing the work of the entire count) 


( nty Superintendent George O. Smith of Bureau County is not 


ee believer that educational tests will be a big factor in the solution of fut 


problems but he is also anxious that his teachers be prepared to partici] 
‘ in the use of ther \s a result the writer offered work on educational tests t 


m ay in the August institute. The Illinois Examination and Teachers’ H 


ra were made the basis of instruction. Two sections of about one hundred te 
| were formed. The work of each section was similar. Consultation period 


provided. The work was given under the following divisions 

1. Each teacher was given the test in exac tly the same manner in wh 
Rt expected to give it 
«. Each teacher was taught how to score the papers using her own as 
»» *ne interpretation of mental age and intelligence quotient in tert 


he child was located was exp 


4. The interpretation of ac} evement ages and achievemer 


ce refe eto] ing suitable remedies for unsati 
After t te é ere asked to make out record sheets like t 
low. These ar € lor reference at any time pupil promoti c 
TI ar It ar} ed for I r3 
: for « t t ealt ind three poor lt I é 
: is t rst Che remainder of the sheet is fille r 
fron 
M tests because we do not follow up. In the case of B 


e follow-up work was carefully planne 


istricts for the « onvenience oi t 
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ae out. The entire county was divided into dT 


arithmetic pupils were generally weak in the fundam 


heet of problems showing the different al 


teachers’ meeting was held in each 
reviewed and the typical type errors called to the 
teachers were urged to secure a textbook 
in detail. The arithmetical errors we arranged as to ty] 
this condition drill cards* in arithmetic were secured. The tea 
ted to go over the test papers « arefully and 
» he did not make satisfactory scores. 


This same idea was carried out in all | 
readily seen that in this way the pupil received indivia 


4 to drill or work on material that he could handle s l}y 


Iccessiully. 


rade eee for the Princeton city 


de standards in the case of gen eral intelligence. This means that 
the different grades of the Princeton city s hools have, on the 
ume capacity to learn as that exhibited by the avera yrresp 
ices This does not n ean that every boy r girl » ft tan 
veneral intelligence. We have some pupils who « fer from it | 
1 ages rar ro ine to eights 


irs. In grade vm we have mental a: 
t range is found in the third grade 
median scores for silent reading were up to the 
ther tests that have been given in the la 


ad te nrohahli ry nre 
schools and 1s probably que in Measure 


f the subject. 
Measuring the results of teaching. New Yort Houghton Mifflin 
rk. M.G. Arithmetic Habituated, Series A 


he attention of the te 


ilities required in the funda 


schools are approximately eq 


tal operations 


her was 


h district At the meeting the test 


phases of the work in arithn 


was 


ners 


1 that would deal with the subject 
ve. It was found 


To 


let each child work on those 
Each teacher is also provided 
nental 
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ividual attention and was 
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he median scores in arithmetic are not satisfactory to the teachers p 
up to the standards of the examination. There are several known causes | 

1. The test involved, in certain grades, operations that are not intrody 
Princeton city schools until later in the life of the child 


2. Previous to the present school year the arithmetic text was not satisfactory 
practical 
3. The instruction was not suitable as it was based entirely on the text 
Che first of these causes cannot be overcome in our course of study 
has been remedied by a change in arithmetic textbooks. The third is being 
as rapidly as possible. At present some improvement has been shown. 
intendent has gone over the pupils’ papers with the classroom teacher and thee 
have been checked. Drill is being given each day on the basis of the individual ; 
Che use of the Illinois Examination in Bureau County and the Prin 
schools has suggested the following observations with reference to its value 
lhe Illinois Examination is simple enough in its operation to make it a \ 
aid to both rural and city schools. It is very easy to administer. 


Teachers who have used it are inclined to diagnose more carefully al! 
failure to do good school work. 


ases 

Teachers, upon discovering that this work is constructive and not for the ; 
of finding fault with teachers, « ooperate very willingly. 

While the examination is standardized for the group and was not used for on 
motional purposes it has designated certain children as worthy of larger opp 
to make progress. 

A better understanding of school situations is developed between tea 
supervisors. 

The examination forms a preliminary basis for the study of gradation or groupin 
by mental rather than chronological ages. 

It will be no small factor in making teachers realize that there is a scientific side 
to teaching that cannot be neglected. 
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National Assuriation of Directors of 


Educational Research 


(E. J. AsupauGH, Secretary and Editor) 


DOES EDUCATION PAY FOR ITSELF? 


tly at an educational meeting, a university professor spent forty-five minutes 

g the proposition that the public should be taxed to support only those 

f education which clearly return full economic value to society and that the 
should pay in full for those other forms of education which do not make such 
He upheld the public elementary schools in general though insisting that 
irse should be changed in such a manner that they might well be called schools 
He suggested that perhaps these schools should carry the youth to 

ize of sixteen by which time the principles of government and economic law 
» well grounded that the children so taught would hence forth be good citi- 


i public support for the graduate schools with emphasis upon the 
ratories, holding that they had already demonstrated that they return 
e for the investment made by public taxation. Concerning the two remaining 


1 yrsed 


f education which are now largely supported by public taxation, secondary 
’ rraduate higher, he had serious doubts of the propriety of continued support. 
He expressed these doubts in such a manner that his hearers recognized that he had no 
s own mind. He was quite sure that the public could not continue the 
trend of publicly supported educational opportunity for ever-increasing 
ers, and that high-school and college education does not pay for itself in returns 

ety at large; perhaps not even in returns to the individual receiving it 
[ am not concerned in this column with a debate of the subject. I do not conceive 
that this is the place or the time for either a rebuttal of his argument or a further 
elaboration of the correctness of his position. But I do believe that the thought 
expressed may legitimately be used as a basis of serious thought and comment apropos 

the work of members of bureaus of educational research. 

Let us face the question: not does education pay for itself but does my work pay 
{? Is the city or state for which I am working getting an adequate economic 
rn for the expenditure which I am causing it to incur? Will this bit of investiga- 
that bit of research, these tabulations, those data gathered and interpreted yield 
a return in financial savings or increased efficiency in the system which can be made 
apparent to the school administration and the public in general? There are scores and 
scores of problems which have not been solved. There are many, many questions to 


which we must now answer that we do not know. There are all sorts of interesting 
things which may be found out about schools and education. But it is quite probable 
that in the present period of financial depression and restless questioning, the public— 


the paying public—is but little interested in pure research, in the answer to idle 
)r in the solution of problems which do not appeal to it and which you may 
t have demonstrated to be of real vital worth. 


75 


| 
| 
‘al. 4 | 
| 
Blactory we | 
rf 
| 
Sune. Re 
needs 
| their 
ases about 
DUPDoge tens 
He 
tunities f 
ers and 
ne side q 


Me re TT 
Me l re 
Cit \ 
For 
nstr 
Conetr 
{ tr 


nt of product in high schools 


of intall 


VAL EDUCATIONAL RES} 1RCH 


thas 
r [ ar t r 
‘ 
i 
tat 1 it be 
ed, 1s It bet er € t-year-old r 
re () 11 months i: 
r ere vee 7 ear r 
ige norms on either of these bases t better t 
ear gre ip 
rge of the research work in the Des Moines, Iowa 
ram of his work there. There is space for but a mer 


easurement of intelligence 
easurement of arithmetic 
surement in three schools in terms of Edu 


etc. of test for incoming teachers 
of obiectiv e ¢ omposition scale 
ot scales, typewriting test 


f intelligence test for kind rgarten children 


nt in the ethics of school children 


f ral ile line in Ie 
of oral and silent reading in grades I! 


I, and m 


ent in the pedagogy of shorthand 
e statistics on athletic records 
tistics and measurements for principals and supervisors 
r f difficulties 
th teachers whenever information ; r advice wi 
wit ervisors 
ia e and reasonable salary schedule 
nent of tables ind of tech whict \ 
er tine 
the State University of Iowa has alled a week 


not planned to attend summer school t 


lucational Qu 


rence of incoming freshman class in one Hi 
easurement of handwriting 


3 + 
‘ 
i 
‘ 
r wit f 
2 } 
oe If one standa 
a 
t 
a? 
in vear gr rin 
rea 
Dr Fr enir 
in a twenty-point pri 
of the main inte 
1 vida r 
> Cit vide + 
it vide 
3. Intensive me 
4 otient 
i 
9 
by 10, 
13 \r experir 
Comparat 
1S. Course it 
16. H in d 
17. Meeting 
ers 
( 
Russell of 
pt cents wi 
~ 


ese 


RESEARCH ASSOCIATION 77 


eof Julvt t 
the recent ere Ss t 
s proble 
the sta ir ymmuttee reported at the 
l The re t of the 1 Statistica 1et 
I in the absence of the writer Dr. Tr in L. Kelle This report 


OF THE SUBCOMMITTEE ON STATISTICAL METHODS OF THE 
STANDARDIZATION COMMITTEE (TENTATIVE 


‘here is a variety of procedure in the treatment of test and measurement data. 

rtunate from the standpoint of ready interpretation of published studies, 

ative of the virility of the movement. The number of workers each 

in the superior merit of his method of presentation makes it undesirable, 

t more information than is now available to your committee to take issue 

of the methods employed, except as implied in proposals (1) and (2). For the 

tated and because statistical procedure in psychology and education should 

with that already established in the older biological and physical sciences your 
e thinks it desirable to offer general rather than specific proposals 

worker presenting a new procedure should definitely recognize that it is 

le to prove the superiority of his method by reporting data interpreted by 

his method alone. Superiority is a relative matter and it is necessary to 

1 method with alternative methods before its superiority can be established. 

Any worker presenting a new procedure should definitely expect that the 

proof lies with him. This implies that he should consider himself responsible 

1) that his method is more reliable in the sense that the probable errors of 

by it are less than those obtained by the more standard methods. 


would give the method prior claim to excellence and it would then devolve 
laimants to establish the superiority of their methods. Or, (b) that the 
1dvantages of interpretation or of expedition that more than compensate 
ened reliability. This implies that the amount of this lessening in reliability 
ind made known by the worker 

ving suggestions as to various measures are offered 
Ot the measures of central tendency the arithmetic average or mean is easily 
1, easily calculated, and usually for such distributions as are found in edu- 
search work, has a smaller probable error than the median or mode. The 
is proved by educational studies of the last decade, is calculated in many 
Its probable error is almost always greater than that of the mean, and has 
een reported. Its brevity (not its accuracy) of calculation and ease of inter- 
recommend it but in case these are not prime considerations your committee 
is the use of the mean in place of the median or mode as a measure of central 
the measures of dispersion the standard deviation usually has, for such 
ms as are found in educational studies, a smaller proportionate probable error 
average deviation or the quartile deviation. Its general use in scientifx 


ations as a measure of dispersion is recommended. For popular presentation 


J 
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0.6745 times the standard deviation, or the probable error, is recommended. [; neces . om 
sary considerations have lead to the use of the median instead of the mean as a mean» 
of central tendency, it is recommended that a measure of dispersion based up 


o 
percentiles, e. g. the quartile deviation,the 10-90 percentile range, etc., be used as + a 
b 
L 


measure of dispersion. 

5. Of the measures of correlation, the Galton-Pearson product moment 
of correlation is well established, having, in the case of rectilinear relati 
smallest probable error of any of the coefficients of correlation. The Spearmap , for 
coethcient based upon the squares of the differences in rank holds a similar | § 
the case of ranked data. These two coefficients are recommended because of the } 
known properties and particularly because their probable errors are a 
measure of relationship for which a formula giving the probable error is not a i 
should be considered as merely in the experimental stage. 1 
levices have been proposed for the measurement of non-rectilinear : 
ships. No recommendation with reference to them is offered. 

6. As a general statement of policy to be followed in educational resear 
committee would recommend that the most reliable of several available 1 res 
xample, as measures of central tendency, of dispersion, of corre 
frequency, etc.) be used, and that when one departs from this rule he s! 
incumbent upon himself to justify his procedure upon substantial grounds, 
necessity for using a less time consuming method, greater simplicity of interpret 
et Truman L. K 
Chairman, Subcommittee on Statistical M ethods 


At the closed meeting of the association at Atlantic City on February 
f the standardization committee appointed at the Ck 
meeting, presented material gathered from the questionnaire which he had sent 
to the members. On the following Thursday he made the report of the committe: 
the report was ordered printed. We present it herewith. 
REPORT OF THE STANDARDIZATION COMMITTEE 
abulation of the returns from the questionnaire sent to members revealed tw » 
outstanding facts: (1) a practically unanimous sentiment in favor of the publication of met 
an ofhcial list of terms, procedures, etc. and (2) a fear that the attempted standardizs- 
tion may be premature. Therefore the statements following have been pre} 
as the recommendation of the committee, but it is suggested that the list be print: 
in the JOURNAL OF EpvucCaTIONAL RESEARCH as tentative standardizations for trial 
purposes only, final approval to be deferred until the annual meeting in 1922. During 
the interval use of the approved forms is to be experimental and voluntary, bu' ; 
members preparing material for publication are asked to try to conform to these = 7 
regulations 
1. Members of the association preparing material for publication, and wishing to - 
use alternative forms or definitions are asked to give reasons or explanations in foot- as 
notes. 


2. A test composed of elements of uniform difficulty, or of several cycles of zs 


uniform difficulty, and used to determine the rate at which the work is done, should ae 
be called Rate Tests. (Jl/ustrations: Cleveland Arithmetic Tests, Starch’s Reading ti 
Tests, Courtis Writing Tests, etc.) pier 
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rhe term “scale” should be applied only to tests or material graded in difficulty, 

i used to measure degree of difficulty or quality. | 

t composed of elements graded in difficulty and used to determine the most / 

F + sect material the subject can handle successfully, under the prescribed condi- 
b 
L 


id be called a performance scale. (Jlusirations: Trabue’s Completion Test, 
Scales, Gray’s Oral Reading Scale, Thorndike’s Reading Scale.) 
ver the objective product of a pupil’s test is measured as to quality by 
th samples of known value, the collection of samples of known value 


. lled a Product Scale. (Jilustrations: Thorndike Handwriting Scale, 
} ition Scale, Rugg’s Lettering Scale 
tion with both rate tests and performance scales the score obtained 
; ited in terms of the results of giving the test to age or grade groups. 
" n consists in placing the score upon a scale of difficulty. Where this 
s been made possible, the testing instrument itself 7 often called a 
rations: Burgess Reading Scale,Ayres Spelling Scale, Hahn-Lackey Geog- 
; et: At this time the committee is not able to recommend a suitable 
s of this type, or for other tests and scales which are mixtures or combi- 
the simpler forms named above. 
term “standard” should be applied to scores as an adjective, and only 
res are set up as goals or objec tives of teaching effort If adopted by an 
ve organization, they may be called “standard scores” without qualification 
et up as goals by an author should be designated as “author’s standard 
\ score representing an actual performance of a group should be called 
Whenever norms are published they should be accompanied by statements 
ber of cases, upon which each norm is based and of the statistical pro edure 
was derived. 
lhe term “average” should be used only in its generic sense to include any and 
s of central tendency. The term “arithmetic mean’”’ should be used as the 
e sum of a series of measures divided by the number of measures. (J//us 
ralior correct use of term: Averages in common use are the mean, the median, and 


[he term “mean” when used alone should be taken to connote the “arith- 
‘mean. All other types of means should be used with descriptive adjectives, 
harmonic mean, geometric mean, etc. 
10. There is need to distinguish between measures of central tendency derived 
from actual scores and those derived from frequency distributions. For instance, the 
rm “‘midscore”’ is suggested as a name to be applied to the middle most measure 
n of the two middle measures when there are two), while the term “median” 
s | be reserved for the value derived from a frequency distribution by interpolation. 
See Rugg’s Statistical Methods A pplied to Education, pages 103-14.) 
11. The term “Performance” should be used to connote the score obtained in a 
test. When, however, there is a desire to convey the idea of comparison with norms 
r standards the term “achievement” should be used. The two terms are, thus, not 
precisely synonymous. 
12. The score of an individual in a test should not be considered as a measure of 
s ability but a3 a record of his performance under the particular conditions under 
which the test was given. Judgments as to ability are always inferences from perform 
ance and the greater the number of trials or tests from which ability is inferred, the 
greater the probable correctness of the inference. 
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t t e rest ted so that it may co to 

t r es of behavior. 
r r the roduct of experience ac ting I Dacity 
15. T I r t for the use of measurement in education has ha r its one 
sa 
the attainment niformity and certainty of interpretation. Great ady nces haw 
alr but still greater advances are possible. To aid in brir oy 
improvement, makers of tests should be careful to inve stigate the effect of ever factor 
involved {n standardization, conforming so far as possible to the law of t single 


variable. In general the process of standardization of a test will involve the followigg 
operations (although not necessarily in the order given) : 
A. Pre paration and selection of test material 
B. Experimental organization of test and instructions for giving the test. 
C. Trial of tentative test to determine value of elements, gross y alidity, rel. 


bility, and optimum conditions of giving, scoring, etc. 


D. Final organization of test 
E. Final formulation of conditions under which test is to be given, scored, 


tabulated, and interpreted. 


F. Official determination of validity 
G. Official determination of reliability 
H. Official determination of norms. 
The association accordingly invites makers of new tests to file with its standardiz. 
tion‘committee for publication data which throw light upon 
\. What a test measures 
B. How reliably it measures. 
C. The principles of formulae of construction. 
D. Statements of conditions for giving, scoring, and tabulating the test and 
for interpreting the results. 


E. Norms 

16. Members of the association and others are invited to send to the committee Kore: 
suggestions for the modification of the above, or other definitions or practices which : 
should be stand rdized 

PROBLEMS 

['wo of the most important types of problems in measurement are those connected 
with the determination of what a test measures, and of how consistently it measures, 
lhe first should be called the problem of validity, the second, the problem of reliability, 

Members are urged to devise and publish means of determining the relation 
between the scores made in a test and other measures of the same ability: in other : 
words, to try to solve the problem of determining the validity of a test. ; me 


Members are also urged to study carefully the relations between the scores made 
in a test and the scores made in other trials of the same or equivalent editions of a test 
the problem of relial ility 

Norms should probably be set only after a careful study of the effects of variation 


of each of the factors of age, grade, time of year, heredity, maturity, training, and 


social status. As rapidly as proves feasible correction tables for variations in each 0 
these factors should be worked out and published. 
Respectfully submitted, B. R. Bu kinGHAM, W. A. McCa t, A. S. Oris, H. 0. 


M. R. 5. A. Courtis, Chairman, Special Standardization Comm: 
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